Transmission system and program

ABSTRACT

A recording medium stores a program which, when executed by a computer, causes the computer to perform a process, the computer connected to a transmission terminal connected to a display device and connected to a second transmission terminal via a network, the process including: displaying an area of a program component on a screen of a second display device connected to the computer; receiving selection of one of display data of a whole area of the screen and display data of the area of the program component; acquiring the display data of the whole area or the area; receiving resolution information indicating a resolution of the display device from the transmission terminal; determining a resolution of the display data based on the resolution information and second resolution information indicating a second display device resolution; and transmitting the display data with the determined resolution to the transmission terminal.

TECHNICAL FIELD

The present invention relates to a program which is executed by acomputer connected to a first transmission terminal which is connectedto a display device and also connected to a second transmission terminalvia a network to perform display data transmission to the secondtransmission terminal.

BACKGROUND ART

A transmission system to conduct videoconferences between remote placesvia a network, such as the Internet, is becoming increasingly popular.In this transmission system, a videoconference is conducted byexchanging image and audio data between two or more transmissionterminals. With recent development of broadband environments,transmission and reception of high-quality image and audio data isenabled and a videoconference in appropriate conditions closelyresembling reality may be conducted between the remote places.

There is a case in which, when conducting a videoconference using atransmission terminal, a person concerned as an attendant of thevideoconference employs an external input device, such as a PC (personalcomputer), in addition to the transmission terminal connected to adisplay device and also connected to another transmission terminal via anetwork, an image, such as meeting documents, is displayed on a displayof the external input device discussed with the partner of thevideoconference. In such a case, according to the related art, theexternal input device has transmitted display data of the image,currently displayed on the display of the external input device, to theother transmission terminal of the communication partner via, thenetwork.

The attendant of the videoconference may share with the communicationpartner not only the image data and the audio data exchanged between thetransmission terminals but also the display data of the image currentlydisplayed on the display of the external input device.

In a case of an external input device which is capable of performingmultitask operations, two or more images of areas (windows) generated bydifferent tasks may be simultaneously displayed on a display device. Inthis case, providing a user with a capability of selecting the displaydata of one of the two or more images to be transmitted to the othertransmission terminal of the partner of the videoconference would beconvenient for the user, and therefore a technology for allowing theuser to select the display data of one of the images is proposed. Forexample, see Japanese Laid-Open Patent Publication No. 2011-254453.

FIG. 40 shows an example of a screen displayed on a display device of anexternal input device disclosed in Japanese Laid-Open Patent PublicationNo. 2011-254453. As shown in FIG. 40, according to Japanese Laid-OpenPatent Publication No. 2011-254453, a user may select one of two areas(a) and (b) of the screen appearing on the display device with a mousecursor (c) and share the display data of the selected area of the screenwith a communication partner of the video conference on the othertransmission terminal.

However, Japanese Laid-Open Patent Publication No. 2011-254453 is silentas to switching between a state in which display data of a whole screenarea of the screen is shared and a state in which display data of aspecific area of the screen is shared. Hence, there is a problem thatthe user is unable to switch from the state in which the display data ofthe specific area is shared to the state where the display data of thewhole screen area is shared (or vice versa).

SUMMARY

In one aspect, the present invention provides a transmission system inwhich a user on a transmission terminal may select arbitrarily one ofsharing of whole area display data and sharing of specific area displaydata such that the selected display data are shared with a communicationpartner of a videoconference on the other transmission terminal.

In one embodiment, the present invention provides a non-transitorycomputer-readable recording medium storing a program representing asequence of instructions, the program which when executed by a computer,causes the computer to perform a process, the computer connected to afirst transmission terminal which is connected to a first display deviceand connected to a second transmission terminal via a network, theprocess including: displaying an area of a program component on a screenof a second display device connected to the computer; receivingselection of at least one of display data displayed in a whole area onthe screen of the second display device and display data displayed inthe area of the program component on the screen as an operation targetof a user; acquiring the display data of the whole area or the areaindicated by the received selection; receiving first resolutioninformation indicating a resolution of the first display device from thefirst transmission terminal; determining a resolution of the acquireddisplay data based on the received first resolution information andsecond resolution information indicating a resolution of the seconddisplay device; and transmitting the acquired display data with thedetermined resolution to the first transmission terminal.

Other objects, features and advantages of embodiments will be apparentfrom the following detailed description when read in conjunction withthe accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing the overall configuration of a transmissionsystem according to an embodiment.

FIG. 2 is a diagram showing of the hardware configuration of atransmission terminal in the transmission system according to theembodiment.

FIG. 3 is a diagram showing the hardware configuration of a transmissionmanagement system in the transmission system according to theembodiment.

FIGS. 4A and 4B are block diagrams showing the functional configurationof a transmission terminal, a relay device, and a transmissionmanagement system which constitutes the transmission system according tothe embodiment.

FIGS. 5A, 5B and 5C are diagrams showing image quality of image data.

FIG. 6 is a diagram showing an example of a change quality managementtable.

FIG. 7 is a diagram showing an example of a relay device managementtable.

FIG. 8 is a diagram showing an example of a terminal authenticationmanagement table.

FIG. 9 is a diagram showing an example of a terminal management table.

FIG. 10 is a diagram showing an example of a destination list managementtable.

FIG. 11 is a diagram showing an example of a session management table.

FIG. 12 is a diagram showing an example of a quality management table.

FIG. 13 is a block diagram showing the functional configuration of anexternal input device.

FIG. 14 is a diagram showing an example of a resolution managementtable.

FIG. 15 is a sequence diagram for explaining a process performed tomanage state information sent from relay devices to the transmissionmanagement system, the state information indicating respective operatingstates of the relay devices.

FIG. 16 is a sequence diagram for explaining a preparatory processperformed before starting communication between the transmissionterminals.

FIG. 17 is a sequence diagram for explaining a process performed by atransmission terminal to establish a session with another transmissionterminal.

FIG. 18 is a sequence diagram for explaining a process performed totransmit display data from an external input device of a transmissionterminal to another transmission terminal of a communication partner anddisplay an image of the display data thereon.

FIG. 19 is a diagram showing an example of display data displayed on adisplay.

FIG. 20 is a diagram showing a screen displayed on a display of anexternal input device and screens displayed on the transmissionterminals in the transmission system.

FIGS. 21A, 21B and 21C are diagrams showing an example of a desktopscreen displayed on an external input device.

FIG. 22 is a sequence diagram for explaining a process of resolutionconversion performed by the external input device and resolutionchanging performed by the transmission terminal when a non-compressedform is used.

FIG. 23 is a flowchart for explaining a process in which a display dataacquiring unit acquires whole area display data or specific area displaydata.

FIG. 24 is a flowchart for explaining a process to convert a resolution.

FIGS. 25A and 25B are a flowchart for explaining a process performed bythe transmission terminal to control a resolution (including changingthe resolution).

FIGS. 26A and 26B are diagrams for explaining a determination step atstep S102 in the process of FIG. 25A.

FIG. 27 is a sequence diagram for explaining a process of resolutionconversion performed by the external input device and resolutionchanging performed by the transmission terminal when a compressed formis used.

FIGS. 28A, 28B and 28C are diagrams showing an example of a desktopscreen displayed on an external input device.

FIG. 29 is a flowchart for explaining a process in which a display dataacquiring unit acquires display data.

FIGS. 30A and 30B are block diagrams showing the functionalconfiguration of an external input device.

FIG. 31 is a diagram showing an example in which a mouse cursor icon isincorporated in display data.

FIG. 32 is a diagram for explaining incorporation of a mouse cursor iconin display data.

FIG. 33 is a flowchart for explaining a process in which a display dataacquiring unit incorporates a mouse cursor icon in display data.

FIGS. 34A, 34B and 34C are diagrams showing examples of areas (windows)which are not to be shared in Windows®.

FIG. 35 is a block diagram showing the functional configuration of anexternal input device.

FIG. 36 is a diagram showing an example of an exception managementtable.

FIG. 37 is a flowchart for explaining a process in which a display dataacquiring unit acquires a whole area display data or a specific areadisplay data.

FIG. 38 is a diagram showing an example of an exception managementtable.

FIG. 39 is a flowchart for explaining a process in which the displaydata acquiring unit acquires a whole area display data or a specificarea display data.

FIG. 40 is a diagram showing an example of a screen displayed on anexternal input device according to the related art.

DESCRIPTION OF EMBODIMENTS

A description will be given of embodiments with reference to theaccompanying drawings.

FIG. 1 is a diagram showing the overall configuration of a transmissionsystem 1 according to an embodiment. The overall configuration of thetransmission system 1 according to the embodiment will be described withreference to FIG. 1.

There are various types of transmission systems. One example may be adata providing system. In the data providing system, content data aretransmitted from one terminal to another terminal via a transmissionmanagement system in a one-way direction. Another example may be acommunication system. In the communication system, information oremotional expressions are mutually communicated among two or moretransmission terminals via the transmission management system. Thecommunication system is utilized for mutually exchanging information oremotional expressions among the two or more communication terminals(i.e., the transmission terminals) via a communication management system(i.e., the transmission management system). Examples of thecommunication system include a videoconference system, a videophonesystem, an audio conference system, a voice-call system, or a personalcomputer screen sharing system.

The following embodiments describe the transmission system, thetransmission management system, and the transmission terminal byconceptualizing the videoconference system as an example of thecommunication system, the videoconference management system as anexample of the communication management system, and the videoconferenceterminal as an example of the communication terminal. That is, thetransmission terminal and the transmission management system utilized inthe embodiments may be applied not only to the videoconference system,but may also be applied to the communication system or the transmissionsystem.

As shown in FIG. 1, the transmission system 1 includes two or moretransmission terminals (10 aa, 10 ab, . . . , 10 db), displays (120 aa,120 ab, . . . , 120 db) for the transmission terminals (10 aa, 10 ab, .. . , 10 db), two or more external input devices (40 aa, 40 ab, 40 db)respectively connected to the transmission terminals (10 aa, 10 ab, . .. , 10 db), two or more relay devices (30 a, 30 b, 30 c, 30 d), atransmission management system 50, a program providing system 90, and amaintenance system 100.

Note that in this embodiment, any one of the transmission terminals (10aa, 10 ab, . . . , 10 db) may be referred to as “transmission terminal10”, any one of the displays (120 aa, 120 ab, . . . , 120 db) may bereferred to as “display 120”, any one of the external input devices (40aa, 40 ab, . . . , 40 db) may be referred to as “external input device40”, and any one of the relay devices (30 a, 30 b, 30 c, 30 d) may bereferred to as “relay device 30”. The display 120 is an example of thedisplay of the transmission terminal 10. Note that in the following, thetransmission terminal and the transmission management system may also besimply referred to as the “terminal” and the “management system”,respectively.

The transmission terminal 10 is configured to transmit and receive imagedata, voice data, etc. to and from another transmission terminal 10.Note that in this embodiment, a case in where the image data aremoving-picture image data will be explained. However, the transmissionterminal 10 may be configured to transmit and receive still-pictureimage data. Alternatively, the image data may include bothmoving-picture image data and still-picture image data. The relay device30 is configured to relay the image data and the voice data among thetransmission terminals 10. The transmission management system 50 isconfigured to manage the transmission terminal 10 and the relay device30 in an integrated manner.

The external input device 40 is connected to the transmission terminal10 and configured to transmit display data, representing an imagedisplayed on a display (“display 216” in FIG. 3) of the external inputdevice 40, to the transmission terminal 10 for displaying the image ofthe display data.

Routers (70 a, 70 b, . . . , 70 f) shown in FIG. 1 are configured toselect optimal paths for the image data and the voice data. Note that inthis embodiment, any one of the routers (70 a, 70 b, . . . , 70 f) maybe referred to as “router 70”. The program providing system 90 includesa not-illustrated hard disk (HD) configured to store programs for aterminal, with which the transmission terminal 10 may implement variousfunctions or various units, and transmit the programs to thetransmission terminal 10. The HD of the program providing system 90 isconfigured to further store programs for a relay device, with which therelay device 30 implements various functions or various units, andtransmit the programs for the relay device to the relay device 30. Inaddition, the HD of the program providing system 90 is configured tostore transmission management programs, with which the transmissionmanagement system 50 implements various functions or various units, andtransmit the transmission management programs to the transmissionmanagement system 50.

As shown in FIG. 1, the transmission terminal 10 aa, the transmissionterminal 10 ab, the relay device 30 a, and the router 70 a are connectedvia a LAN (local area network) 2 a so that they may communicate witheach other. The transmission terminal 10 ba, the transmission terminal10 bb, the relay device 30 b, and the router 70 b are connected via aLAN 2 b so that they may communicate with each other. The LAN 2 a andthe LAN 2 b are connected by a dedicated line 2 ab including a router 70c so that they communicate with each other, and the LAN 2 a and the LAN2 b are set up in a predetermined area A. For example, the area A may beJapan, the LAN 2 a may be set up in an office in Tokyo, and the LAN 2 bmay be set up in an office in Osaka.

Meanwhile, the transmission terminal 10 ca, the transmission terminal 10cb, the relay device 30 c, and the router 70 d are connected via a LAN 2c so that they may communicate with each other. The transmissionterminal 10 da, the transmission terminal 10 db, the relay device 30 d,and the router 70 e are connected via a LAN 2 d so that they maycommunicate with each other. The LAN 2 c and the LAN 2 d are connectedby a dedicated line including a router 70 f so that they communicatewith each other, and the LAN 2 c and the LAN 2 d are set up inpredetermined area B. For example, the area B may be USA, the LAN 2 cmay be is set up in an office in New York, and the LAN 2 d may be set upin an office in Washington, D.C. The area A and the area B are connectedfrom the routers 70 c and 70 f via the Internet 2 i so that the area Aand the area B are mutually in communication via the Internet 2 i.

The transmission management system 50 and the program providing system90 are connected to the transmission terminal 10 and the relay device 30via the Internet 2 i so that they may communicate with each other.Alternatively, the transmission management system 50 and the programproviding system 90 may be located in the area A or the area B, or maybe located in an area other than the areas A and B.

Note that in this embodiment, a communication network 2 is made up ofthe LAN 2 a, the LAN 2 b, the dedicated line 2 ab, the Internet 2 i, thededicated line 2 cd, the LAN 2 c, and the LAN 2 d. In the communicationnetwork 2 of FIG. 1, the set of four numbers attached under each of thetransmission terminal 10, the relay device 30, the transmissionmanagement system 50, the router 70, and the program providing system 90simply indicates an IP address under the typical IPv4 (Internet Protocolversion 4). For example, the IP address of the transmission terminal 10aa is “1.2.1.3” as shown in FIG. 1. Alternatively, the IP addressesunder the IPv6 may be used instead of the IP addresses under the IPv4.However, in this embodiment, the IPv4 is used for simplifying theillustration.

[Hardware Configuration of Embodiment]

Next, the hardware configuration of the transmission system 1 accordingto the embodiment will be described.

FIG. 2 is a diagram showing the hardware configuration of the terminal(i.e., transmission terminal) 10 in the transmission system 1 accordingto the embodiment. As shown in FIG. 2, the terminal 10 includes acentral processing unit (CPU) 101 configured to control overalloperations of the terminal 10, a read-only memory (ROM) 102 storingprograms for the terminal, a random access memory (RAM) 103 utilized asa work area of the CPU 101, a flash memory 104 configured to storevarious data such as image data or audio data, a solid state drive (SDD)105 configured to control retrieval and writing (storing) of the variousdata in the flash memory 104 and the like based on the control of theCPU 101, a medium drive 107 configured to control retrieval and writing(storing) of data into a recording medium 106 such as a flash memory, anoperation button 108 operated by a user for selecting an address of theterminal 10, a power switch 109 for switching ON/OFF of the power of theterminal 10, and a network IF (interface) 111 for transmitting datautilizing the communication network 2.

The terminal 10 further includes a camera 112 configured to image asubject based on the control of the CPU 101, an imaging device IF 113configured to control driving of the camera 112, a microphone 114configured to pick up audio sound, such as voice, sound or noise, aspeaker 115 configured to output audio sound, a voice input/output IF116 configured to provide an interface for the inputting/outputting ofthe audio sound from the microphone 114 and to the speaker 115, adisplay IF 117 configured to transmit image data to the external display120 based on the control of the CPU 101, an external device IF 118configured to connect various external devices to the connecting ports,an alarm lamp 119 configured to inform the user of various functionalproblems of the terminal 10, and a bus line 110 such as an address busor a data bus for electrically connecting the elements and devices withone another shown in FIG. 2 via the bus line 110.

Note that the recording medium 106 is removable from the terminal 10. Inaddition, if the recording medium 106 is a non-volatile memoryconfigured to retrieve or write data based on the control of the CPU101, the recording medium 106 is not limited to the flash memory 104,and may be an electrically erasable and programmable ROM (EEPROM). Thecamera 112 includes lenses and a solid-state image sensor configured toconvert light into electric charges to produce digital images (videos)of the subject. Examples of the solid-state image sensor include acomplementary metal oxide semiconductor (CMOS) or a charge coupleddevice (CCD). A hard disk drive (HDD) may be used instead of the SSD105.

The display 120 is a display unit formed of liquid crystal or an organicEL material configured to display images of the subject or icons foroperating the terminal 10.

Further, the aforementioned programs for the terminal may be recorded inan installable format or in an executable format on a computer-readablerecording medium such as the recording medium 106 to distribute such arecording medium.

The external device IF 118 may be connected via a universal serial bus(USB) with external devices such as the external camera, the externalmicrophone, and the external speaker. On connecting the external camerato the external device IF 118 via the USB, the external camera is drivenin priority to the built-in camera 112. Likewise, on connecting theexternal microphone or external speaker to the external device 118 viathe USB, the external microphone or external speaker is driven inpriority to the built-in microphone 114 or the built-in speaker 115. Thetransmission terminal 10 may be any of a PC, a smart phone, a tabletterminal, a mobile phone, etc.

FIG. 3 is a diagram showing the hardware configuration of thetransmission management system 50 in the transmission system 1 accordingto the embodiment. As shown in FIG. 3, the transmission managementsystem 50 includes a central processing unit (CPU) 201 configured tocontrol overall operations of the management system 50, a read-onlymemory (ROM) 202 storing programs for the transmission management, arandom access memory (RAM) 203 utilized as a work area of the CPU 201, ahard disk (HD) 204 configured to store various data, a hard disk drive(HDD) 205 configured to control retrieval and writing (storing) of thevarious data in the HD 204 based on the control of the CPU 201, a mediumdrive 207 configured to control retrieval and writing (storing) of datainto a recording medium 206 such as a flash memory, a display IF 208configured to display various information such as a cursor, menus,windows, characters and images, on a display 216, a network IF 209 fortransmitting data utilizing the later-described communication network 2,a keyboard 211 including plural keys for inputting the characters,numerals, and various instructions, a mouse 212 for selecting orexecuting various instructions, selecting items to be processed, andmoving the cursor, a CD-ROM drive 214 configured to control retrieval orwriting of data in the compact disk read-only memory (CD-ROM) 213 as anexample of a removable recording medium, an external device IF 215configured to connect various external devices to the connecting ports,and a bus line 210 such as an address bus or a data bus for electricallyconnecting the elements and devices with one another illustrated in FIG.3 via the bus line 210. The display 216 is an example of a display unitof the external input device 40.

Note that the aforementioned programs for the transmission managementmay be recorded in an installable format or in an executable format on acomputer-readable recording medium such as the recording medium 206 orthe CD-ROM 213 to distribute such a recording medium. The programs forthe transmission management may be recorded on the HD 204.

Further, the external input device 40 includes a hardware configurationsimilar to that of the transmission management system, and thedescription of the hardware configuration of the external input device40 is therefore omitted. Note that the ROM 202 stores programs forcontrolling the external input device 40. In this case, the programs forthe external input device 40 may also be recorded in an installableformat or in an executable format on a computer-readable recordingmedium such as the recording medium 206 or the CD-ROM 213 to distributesuch a recording medium.

Further, the relay device 30 includes a hardware configuration similarto that of the management system 50, and the description of the hardwareconfiguration of the relay device 30 is therefore omitted. Note that theROM 202 stores programs for controlling the relay device 30. In thiscase, the programs for the relay device may also be recorded in aninstallable format or in an executable format on a computer-readablerecording medium such as the recording medium 206 or the CD-ROM 213 todistribute such a recording medium.

Further, the program providing system 90 includes a hardwareconfiguration similar to that of the management system 50, and thedescription of the hardware configuration of the program providingsystem 90 is therefore omitted. Note that the ROM 202 stores programsfor controlling the program providing system 90. In this case, theprograms for the program providing system 90 may also be recorded in aninstallable format or in an executable format on a computer-readablerecording medium such as the recording medium 206 or the CD-ROM 213 todistribute such a recording medium.

Further, the maintenance system 100 includes a hardware configurationsimilar to that of the management system 50, and the description of thehardware configuration of the maintenance system 100 is thereforeomitted. The maintenance system 100 is a computer configured to maintainor manage at least one of the terminal 10, the relay device 30, themanagement system 50, and the program supply system 90. For example, ifthe maintenance system 100 is domestically located while the terminal10, the relay device 30, the management system 50 or the program supplysystem 90 is located abroad, the maintenance system 100 remotelymaintains or manages at least one of the terminal 10, the relay device30, the management system 50, and the program supply system 90 via thecommunication network 2.

Further, the maintenance system 100 may be arranged to maintain, withoutusing the communication network 2, a model number, a production serialnumber, a sale place, maintenance inspection or failure history in atleast one of the terminal 10, the relay device 30, the management system50, and the program providing system 90.

Note that other examples of the aforementioned removable recordingmedium may include a compact disc recordable (CD-R), a digital versatiledisk (DVD), and a Blu-ray Disc (BD).

[Functional Configuration of Embodiment]

Next, the functional configuration of the embodiment will be describedwith reference to FIGS. 4A to 14.

FIGS. 4A and 4B are block diagrams showing the functional configurationof the transmission terminal 10, the relay device 30, and thetransmission management system 50 which constitute the transmissionsystem 1 according to the embodiment. In the example shown in FIGS. 4Aand 4B, the transmission terminal 10, the relay device 30, and thetransmission management system 50 are connected via the communicationnetwork 2 so that they carry out data communications with one anothervia the communication network 2. The external input device 40 isconnected to the transmission terminal 10 to transmit data to andreceive data from the transmission terminal 10. Note that the programproviding system 90 and the maintenance system 100 shown in FIG. 1 arenot directly associated with videoconference communications and theillustration thereof will be omitted in FIGS. 4A and 4B.

FIGS. 5A, 5B and 5C are diagrams showing image quality of image data.FIG. 6 is a diagram showing an example of a change quality managementtable 3001T. FIG. 7 is a diagram showing an example of a relay devicemanagement table 5001T. FIG. 8 is a diagram showing an example of aterminal authentication management table 5002T. FIG. 9 is a diagramshowing an example of a terminal management table 5003T. FIG. 10 is adiagram showing an example of a destination list management table 5004T.FIG. 11 is a diagram showing an example of a session management table5005T. FIG. 12 is a diagram showing an example of a quality managementtable 5007T. FIG. 13 is a block diagram showing the functionalconfiguration of the external input device 40. FIG. 14 is a diagramshowing an example of a resolution management table 4001T.

[Functional Configuration of Terminal]

As shown in FIG. 4B, the transmission terminal 10 includes atransmitting/receiving (TX/RX) unit 11, an operation input receivingunit 12, a login request unit 13, an imaging unit 14 a, a displaycontrol unit 14 b, a display image acquiring unit 14 c, a voice inputunit 15 a, a voice output unit 15 b, a delay detecting unit 17, anexternal information transmitting/receiving (TX/RX) unit 18, astore/read processing unit 19, a resolution acquiring unit 21 a, aresolution determining unit 21 b, a resolution selecting unit 21 c, aresolution changing unit 21 d. These units represent functions or unitsimplemented by any of the elements and, devices shown in FIG. 2, whichare activated by the instructions from the CPU 101 based on the programsstored in the ROM 102. Further, the transmission terminal 10 includes astorage unit 1000 formed of the SSD 105 shown in FIG. 2.

[Functional Units of Terminal]

Next, the functional units of the transmission terminal 10 will bedescribed. The transmitting/receiving unit 11 of the terminal 10 isconfigured to transmit various data to and receive such data from othertransmission terminals 10, the relay device 30 and the transmissionmanagement system 50 via the communication network 2. The function ofthe transmitting/receiving unit 11 may be implemented by the network IF111 shown in FIG. 2. The operation input receiving unit 12 is configuredto receive various inputs from a user. The function of the operationinput receiving unit 12 may be implemented by the operation button 108and the power switch 109 which are shown in FIG. 2. For example, whenthe user switches ON the power switch 109 shown in FIG. 2, the operationinput receiving unit 12 (FIG. 4B) receives a power-ON signal to switchON the power of the transmission terminal 10. Further, the operationinput receiving unit 12 receives the resolution information indicating aresolution which is input by a user operation.

The login request unit 13 is configured to automatically transmit loginrequest information, indicating that the user desires to login, and acurrent IP address of the terminal 10 as an IP address of a requestsource terminal from the transmitting/receiving unit 11 to thetransmission management system 50 via the communication network 2 whenthe power-ON signal is received by the operation input receiving unit 12as a trigger. The function of the login request unit 13 may beimplemented by the instructions from the CPU 101 shown in FIG. 2.

The imaging unit 14 a is configured to take an image of a subject andoutput image data of the subject. The function of the imaging unit 14 amay be implemented by the camera 112 and the image sensor IF 113 shownin FIG. 2. The display control unit 14 b is configured to performrendering of image data and control the displaying of an imagerepresented by the image data on the display 120. The function of thedisplay control unit 14 b may be implemented by the display IF 117 shownin FIG. 2. The display image acquiring unit 14 c is configured toacquire image data representing an image currently displayed on thedisplay 120. In this embodiment, data representing an image which istaken by the camera 112 will be referred to as “image data” and datarepresenting an image currently displayed on the display 216 of theexternal input device 40 will be referred to as “display data”. Forexample, the image data and the display data are in a form conforming toany of JPEG (Joint Photographic Experts Group), Bitmap, GDI (GraphicsDevice Interface), etc.

The voice input unit 15 a is configured to input user's voice, convertthe voice into an audio signal, and output audio data of the audiosignal. The function of the voice input unit 15 a may be implemented bythe microphone 114 and the voice input/output IF 116 shown in FIG. 2.The voice output unit 15 b is configured to convert the audio signalaccording to the voice data into voice and output the voice. Thefunction of the voice output unit 15 b may be implemented by the speaker115 and the voice input/output IF 116 shown in FIG. 2.

The delay detecting unit 17 is configured to detect a delay time (ms) ofimage data or voice data transmitted from another transmission terminal10 via the relay device 30. The function of the delay detecting unit 17may be implemented by the instructions from the CPU 101 shown in FIG. 2.The external information transmitting/receiving unit 18 is configured totransmit data to and receive data from the external input device 40 viathe external device IF 118 shown in FIG. 2. The store/read processingunit 19 is configured to store various data in a storage unit 1000 orread the various data stored in the storage unit 1000. The function ofthe store/read processing unit 19 may be implemented by the SSD 105shown in FIG. 2.

The resolution acquiring unit 21 a is configured to acquire resolutionswhich can be used by the display 120 connected to the transmissionterminal 10. The resolution determining unit 21 b is configured todetermine whether a resolution other than a resolution of a standardaspect ratio used for transmission of image data is included in theresolutions acquired by the resolution acquiring unit 21 a. Theresolution selecting unit 21 c is configured to select the resolutionother than the resolution of the standard aspect ratio or the resolutionof the standard aspect ratio from among the resolutions which can beused by the display 120 based on a result of the determination by theresolution determining unit 21 b. The resolution changing unit 21 d isconfigured to change the resolution of the display 120 to a resolutionindicated by the resolution information received by the operation inputreceiving unit 12. In the following, the term “resolution” representsthe number of pixels existing per unit area on a display screen and“resolution information” contains a resolution of a vertical directionand a resolution of a horizontal direction.

The storage unit 1000 stores a terminal ID (identifier) for identifyingthe transmission terminal 10, a password, a relay device ID foridentifying the relay device 30 which transmits image data, voice dataand various data, and an IP address of a destination terminal. Further,the storage unit 1000 stores the programs for the external input device(including a display data acquisition program 1451, a display datatransmission program 1452, and a resolution conversion program 1453). Bytransmitting these programs to the external input device 40 andinstalling the programs in the external input device 40, the externalinput device 40 is provided with a display data acquiring unit 451, adisplay data transmitting unit 452 and a resolution converting unit 453,which will be described later. The storage unit 1000 includes aresolution storing unit 1005 which stores resolution informationindicating a resolution of display data which is output or transmittedby the transmission terminal 10.

The external information transmitting/receiving unit 18 in thetransmission terminal 10 according to the embodiment is an example of aresolution information transmitting unit and a display data receivingunit, and the display control unit 14 b is an example of a displaycontrol unit.

Note that the terminal ID and the later-described relay-device ID inthis embodiment individually indicate identification information formedof language, characters, and various symbols that uniquely identify theterminal 10 and the relay device 30, respectively. The terminal ID andthe relay-device ID may be identification information formed of acombination of two or more of the language, characters, and varioussymbols. In the following, the terminal 10 as a request source terminalthat requests initiation of a videoconference communication will becalled a “request source terminal”, and the terminal 10 as a destinationterminal to which the request is sent will be called a “destinationterminal”.

[Functional Configuration of Relay Device]

Next, functions and units of the relay device 30 will be described. Asshown in FIG. 4B, the relay device 30 includes a transmitting/receiving(TX/RX) unit 31, a state detecting unit 32, a data quality check unit33, a changed quality managing unit 34, a data quality changing unit 35,and a store/read processing unit 39. These units represent functions andunits implemented by any of the elements and devices shown in FIG. 3,which are activated by the instructions from the CPU 201 based on theprograms stored in the ROM 202. Further, the relay device 30 includes astorage unit 3000 formed of any of the ROM 202, the RAM 203 and the HDD205 which are shown in FIG. 3.

[Change Quality Management Table]

The storage unit 3000 includes a change quality management DB (database)3001 formed of the change quality management table 3001T shown in FIG.6. In the change quality management table, the IP addresses of theterminals 10 utilized as relay destination terminals are associated withimage quality of the image data relayed by the respective relay devices30.

Herein, the resolutions of the image data treated in the transmissionsystem 1 according to the embodiment will be described. FIG. 5A shows abase image used as a low-resolution image formed of 160×120 pixels. FIG.5B shows a medium-resolution image formed of 320×240 pixels. FIG. 5Cshows a high-resolution image formed of 640×480 pixels. When the imagedata are relayed via a narrow-bandwidth path, the low-resolution imagedata used as the base image are relayed. When the image data are relayedvia a relatively wide bandwidth path, the low-resolution image data usedas the base image and the medium-resolution image data exhibiting mediumimage quality are relayed. When the image data are relayed via anextremely wide bandwidth path, the low-resolution image data used as thebase image, the medium-resolution image data exhibiting the medium imagequality, and the high-resolution image data exhibiting high imagequality are relayed. As in the changed quality management table 3001Tshown in FIG. 6, if, for example, the relay device 30 relays the imagedata to the destination terminal 10 db having an IP address “1.3.2.4”(FIG. 1), the image quality of the image data relayed is “high imagequality”.

[Functional Units of Relay Device]

Next, functions and units of the relay device 30 will be described. Notethat in the following description, the functions of the relay device 30are illustrated in association with the main elements and devices forimplementing the units of the relay device 30 shown in FIG. 3.

The transmitting/receiving unit 31 of the relay device 30 is configuredto transmit various data to and receive such data from othertransmission terminals 10, the relay devices 30, and the transmissionmanagement system 50 via the communication network 2. The function ofthe transmitting/receiving unit 31 may be implemented by the network IF209 shown in FIG. 3. The state detecting unit 32 is configured to detectan operating state of the relay device 30 having the state detectingunit 32. The function of the state detecting unit 32 may be implementedby the instructions from the CPU 201 shown in FIG. 3. The operatingstate may be “ONLINE”, “OFFLINE”, “DURING CALL” or “FAILED”.

The data quality check unit 33 is configured to search the changequality management table (FIG. 6) by the IP address of the destinationterminal 10 as a search key and check the image quality for the relayedimage data by extracting the image quality for the corresponding imagedata to be relayed. The function of the data quality check unit 33 maybe implemented by the instructions from the CPU 201 shown in FIG. 3. Thechange quality managing unit 34 is configured to change content of thechange quality management DB 3001 based on the later-described qualityinformation which is transmitted from the transmission management system50. The function of the change quality managing unit 34 may beimplemented by the instructions from the CPU 201 shown in FIG. 3. Forexample, assume that while a videoconference is held between a requestsource terminal 10 aa having a terminal ID “01 aa” and a destinationterminal 10 db having a terminal ID “01 db”, another videoconference isinitiated between a request source terminal bb and a destinationterminal ca via the communication network 2. In this case, if thereception of the image data is delayed in the destination terminal 10db, the relay device 30 may need to degrade the image quality of therelayed image data from high image quality to medium image quality. Insuch a case, the content of the change quality management DB 3001 ischanged such that the image quality of the image data relayed by therelay device 30 is degraded from the high image quality to the mediumimage quality.

The data quality changing unit 35 is configured to change the imagequality of the image data sent from the request source terminal 10 basedon the changed content of the change quality management DB 3001. Thefunction of the data quality changing unit 35 may be implemented by theinstructions from the CPU 201 shown in FIG. 3. The store/read processingunit 39 is configured to store various data in the storage unit 3000 andread the various data stored in the storage unit 3000. The function ofthe store/read processing unit 39 may be implemented by the HDD 205shown in FIG. 3.

[Functional Configuration of Management System]

Next, functions and units of the transmission management system 50 willbe described. As shown in FIG. 4A, the transmission management system 50includes a transmitting/receiving (TX/RX) unit 51, a terminalauthenticating unit 52, a state managing unit 53, a terminal extractingunit 54, a terminal state acquiring unit 55, a session managing unit 57,a quality determination unit 58, a store/read processing unit 59, and adelay time managing unit 60. These units represent functions and unitsimplemented by any of the elements and devices shown in FIG. 3, whichare activated by the instructions from the CPU 201 based on the programsstored in the ROM 202. Further, the transmission management system 50includes a storage unit 5000 formed of the ROM 202, the RAM 203, or theHDD 205 shown in FIG. 3.

[Relay Device Management Table]

The storage unit 5000 includes a relay device management DB 5001 formedof the relay device management table 5001T shown in FIG. 7. In the relaydevice management table 5001T, relay device IDs of the relay devices 30are associated with operating states of the relay devices 30, date/timesof reception at which the state information indicating the operatingstates of the relay devices 30 is received by the transmissionmanagement system 50, IP addresses of the relay devices 30, and maximumdata transmission speeds (Mbps) of the relay devices 30. For example, inthe relay device management table 5001T shown in FIG. 7, the relaydevice 30 a (FIG. 1) having the relay device ID “111 a” is associatedwith the operating state “ONLINE”, the time/date of reception“2009.11.10.13:00” at which the state information of the relay device 30a is received by the transmission management system 50, the IP address“1.2.1.2” of the relay device 30 a, and the maximum data transmissionspeed “100 Mbps” of the relay device 30 a.

[Terminal Authentication Management Table]

The storage unit 5000 includes a terminal authentication management DB5002 formed of the terminal authentication management table 5002T shownin FIG. 8. In the terminal authentication management table 5002T,passwords are individually associated with the terminal IDs of thetransmission terminals 10 managed by the transmission management system50. For example, in the terminal authentication management table 5002Tshown in FIG. 8, the terminal ID “01 aa” of the terminal 10 aa (FIG. 1)is associated with the password “aaaa”.

[Terminal Management Table]

The storage unit 5000 includes a terminal management DB 5003 formed ofthe terminal management table 5003T shown in FIG. 9. In the terminalmanagement table 5003T, the terminal IDs of the transmission terminals10 are associated with operating states of the transmission terminals10, date/times of reception at which the later-described login requestinformation is received by the transmission management system 50, and IPaddresses of the transmission terminals 10. For example, in the terminalmanagement table 5003T shown in FIG. 9, the terminal ID “01 aa” of theterminal 10 aa (FIG. 1) is associated with the operating state “ONLINE”of the terminal 10 aa, the date/time of reception “2009.11.10.13:40” atwhich the login request information is received by the management system50, and the IP address “1.2.1.3.” of the terminal 10 aa.

[Destination List Management Table]

The storage unit 5000 includes a destination list management DB 5004formed of the destination list management table 5004T shown in FIG. 10.In the destination list management table 5004T, terminal IDs ofdestination terminals 10 registered as destination candidate terminals10 are associated with the terminal IDs of the request source terminal10 that requests initiation of a videoconference communication. Forexample, in the destination list management table 5004T shown in FIG.10, the terminal ID “01 aa” of the request source terminal 10 aa(FIG. 1) is associated with the terminal IDs of the destinationcandidate terminals 10 which the request source terminal 10 aa is ableto request initiation of the videoconference, including the transmissionterminal 10 ab having a terminal ID “01 ab”, the transmission terminal10 ba having a terminal ID “01 ba”, and the transmission terminal 10 dbhaving a terminal ID “01 db”. The destination candidate terminals 10 maybe added or deleted based on the instructions from the request sourceterminal 10 to the transmission management system 50.

[Session Management Table]

The storage unit 5000 includes a session management DB 5005 formed of asession management table 5005T shown in FIG. 11. In the sessionmanagement table 5005T, session IDs that identify sessions in which thecommunication data are transmitted and received between the terminalsare associated with the relay device IDs of the relay devices 30utilized for relaying the image data and the audio data, the terminal IDof the request source terminal 10, the terminal ID of the destinationterminal 10, the delay time (ms) in receiving the image data by thedestination terminal 10, and the date/time of reception at which thedelay information indicating the delay time transmitted from thedestination terminal 10 is received by the management system 50.

For example, in the session management table 5005T shown in FIG. 11, therelay device 30 a (having the relay device ID “111 a”) selected forexecuting the session having the session ID “sel” and relaying imagedata and voice data is associated with the terminal ID “01 aa” of therequest source terminal 10 aa (FIG. 1) that has a videoconference withthe destination terminal 10 db having the terminal ID “01 db”. Further,the session management table 5005T indicates that the relay device 30 arelays the image data and the audio data between the request sourceterminal 10 aa and the destination terminal 10 db, and the delay timewhen the image data are received by the destination terminal 10 db atthe date/time of reception “2009.11.10. 14:00” is 200 (ms).

Note that when the videoconference is conducted between the twoterminals 10, the date/time of reception of the delay information may bemanaged not based on the delay information received at the destinationterminal 10 but based on the delay information transmitted from therequest source terminal 10. However, when the videoconference isconducted among three or more transmission terminals 10, the date/timeof reception of the delay information may be managed based on the delayinformation transmitted from the image data and voice data receptionside transmission terminal 10.

[Quality Management Table]

The storage unit 5000 includes a quality management DB 5007 formed ofthe quality management table 5007T shown in FIG. 12. In the qualitymanagement table 5007T, the delay time (ms) in receiving the image dataat the request source terminal 10 or the destination terminal 10 isassociated with the image quality of the image data relayed by the relaydevice 30.

[Functional Units of Management System]

Next, functional units of the transmission management system 50 will bedescribed. Note that in the following description, the functions of themanagement system 50 are illustrated in association with the mainelements and devices for implementing the units of the management system50 shown in FIG. 3.

The transmitting/receiving unit 51 (FIG. 4A) of the management system 50is configured to transmit various data (information) to and receive suchdata from other transmission terminals 10, the relay devices 30, andother systems (i.e., the program providing system 90 and the maintenancesystem 100) via the communication network 2. The terminal authenticatingunit 52 is configured to search the terminal authentication managementtable 5002T of the storage unit 5000 by a terminal ID and a passwordcontained in the received login request information as search keys, andauthenticate the corresponding terminal 10 based on whether the terminalID and the password contained in the received login request areidentical to those managed in the terminal authentication managementtable 5002T. The state managing unit 53 is configured to manage theterminal management table 5003T (FIG. 9) by associating the terminal IDof the request source terminal 10 that has requested for login with theoperating state of the request source terminal 10, the date/time ofreception of the login request information by the management system 50,and the IP address of the request source terminal 10.

The terminal extracting unit 54 is configured to search the destinationlist management table 5004T (FIG. 10) by the terminal ID of the requestsource terminal 10 that has requested for the login as a search key, andretrieve the terminal IDs of the destination candidate terminals 10capable of communicating with the request source terminal 10. As aresult, the terminal extracting unit 54 extracts the terminal IDs of thedestination candidate terminals 10 capable of communicating with therequest source terminal 10. Further, the terminal extracting unit 54 isconfigured to search the destination list management table (FIG. 10) bythe terminal ID of the request source terminal 10 which has requestedfor the login as a search key, and extract terminal IDs of other requestsource terminals 10 that have registered the above terminal ID of therequest source terminal 10 as a destination candidate terminal 10.

The terminal state acquiring unit 55 is configured to search theterminal management table 5003T (FIG. 9) by the terminal IDs of thedestination candidate terminals 10 extracted by the terminal extractingunit 54 as search keys, and retrieve the operating states of theextracted destination candidate terminals 10 by the terminal IDs of theextracted destination candidate terminals 10. Thus, the terminal stateacquiring unit 55 acquires the operating states of the destinationcandidate terminals 10 capable of communicating with the request sourceterminal 10 that has requested for the login. Further, the terminalstate acquiring unit 55 is configured to search the terminal managementtable 5003T (FIG. 9) by the terminal IDs of the terminals 10 extractedby the terminal extracting unit 54 as search keys, and acquire theoperating state of the request source terminal 10 that has requested forthe login.

The session managing unit 57 is configured to store and manage, in thesession management DB 5005 (the session management table 5005T of FIG.11) of the storage unit 5000, the session IDs generated in associationwith the terminal ID of the request source terminal and the terminal IDof the destination terminal. Further, the session managing unit 57 isconfigured to store and manage the relay device ID of the relay device30 in the session management table 5005T (FIG. 11).

The quality determination unit 58 is configured to search the qualitymanagement table 5007T (FIG. 12) by the delay time as a search key,extract the image quality of the corresponding image data, and determinethe image quality of the image data relayed by the relay device 30. Thestore/read processing unit 59 is configured to store various data in thestorage unit 5000 and retrieve various data from the storage unit 5000.The function of the store/read processing unit 59 may be implemented bythe HDD 205 shown in FIG. 3. The delay time managing unit 60 isconfigured to search the terminal management table 5003T (FIG. 9) by theIP address of the destination terminal 10 as a search key, and extractthe corresponding terminal ID. Further, the delay time managing unit 60is configured to store the delay time indicated by the delay informationin a delay time field of the record corresponding to the extractedterminal ID in the session management table 5005T (FIG. 11).

[Functional Configuration of External Input Device]

As shown in FIG. 13, the external input device 40 includes atransmitting/receiving unit 41, a connection detecting unit 42, aninstallation determining unit 43 a, a program acquiring unit 43 b, aresolution acquiring unit 44, a display data acquiring unit 451, adisplay data transmitting unit 452, a resolution converting unit 453, anoperation input receiving unit 46, a display control unit 47, a mountingunit 48, and a store/read processing unit 49. These units representfunctions and units implemented by any of the elements and devices shownin FIG. 3, which are activated by the instructions from the CPU 201based on the programs stored in the ROM 202. Further, the display dataacquiring unit 451, the display data transmitting unit 452, and theresolution converting unit 453 may be implemented by downloading thedisplay data acquisition program 1451, the display data transmissionprogram 1452, and the resolution conversion program 1453 from thestorage unit 1000 (FIG. 4B) of the terminal 10 by the program acquiringunit 43 b and installing these programs in the external input device 40.Further, the external input device 40 includes a storage unit 4000formed of any of the ROM 202, the RAM 203, and the HDD 205 which areshown in FIG. 3. Although not illustrated in the drawings, the storageunit 4000 of the external input device 40 incorporates an operatingsystem (OS), such as Windows®, the classic Mac® OS, Mac® OS X Lion, Mac®OS X, or the renamed OS X, and, when connected to another device, theCPU 201 of the external input device 40 is capable of executing theprograms for the external input device 40 and the downloaded programs onthe OS.

[Functional Units of External Input Device]

Next, functional units of the external input device 40 will bedescribed.

The transmitting/receiving unit 41 (FIG. 13) is configured to transmitvarious data (information) to and receive such data from thetransmission terminal 10. The function of the transmitting/receivingunit 41 may be implemented by the network IF 209 shown in FIG. 3. Asinformation related to the embodiment, when a display resolution ischanged in the transmission terminal 10, the transmitting/receiving unit41 receives resolution information (first resolution information)indicating the changed resolution from the transmission terminal 10. Theconnection detecting unit 42 is configured to detect that thecommunication of various data between the external input device 40 andthe transmission terminal 10 via the external device IF 215 shown inFIG. 3 is enabled.

The installation determining unit 43 a is configured to determinewhether the display data acquisition program 1451, the display datatransmission program 1452, and the resolution conversion program 1453are installed in the external input device 40 if the connectiondetecting unit 42 detects that the data communication between theexternal input device 40 and the transmission terminal 10 is enabled.The program acquiring unit 43 b is configured to acquire the displaydata acquisition program 1451, the display data transmission program1452, and the resolution conversion program 1453 from the storage unit1000 (FIG. 4A) of the transmission terminal 10 if the installationdetermining unit 43 a determines that the display data acquisitionprogram 1451, the display data transmission program 1452, and theresolution conversion program 1453 are not installed in the externalinput device 40.

The resolution acquiring unit 44 is configured to acquire resolutioninformation (second resolution information) indicating a resolution ofthe display 216 of the external input device 40. The operation inputreceiving unit 46 is configured to receive the information input by auser operation. The display control unit 47 is configured to display onthe display 216 the image read by the later-described store/readprocessing unit 49. The mounting unit 48 is configured to mount thestorage unit 1000 of the terminal 10 on the external input device 40.Thus, the external input device 40 is able to install the display dataacquisition program 1451, the display data transmission program 1452,and the resolution conversion program 1453 in the external input device40.

The display data acquiring unit 451 is configured to acquire displaydata indicating an image currently displayed on the display 216 of theexternal input device 40. The display data transmitting unit 452 isconfigured to transmit the display data acquired by the display dataacquiring unit 451 to the transmission terminal 10. The resolutionconverting unit 453 is configured to determine a resolution of thedisplay data to be transmitted from the external input device 40 to thetransmission terminal 10, based on resolution information (firstresolution information) of the display 120 of the transmission terminal10 received from the transmission terminal 10 and stored in the storageunit 4000 of the external input device 40 and resolution information(second resolution information) of the display 216 of the external inputdevice 40.

The store/read processing unit 49 is configured to store various data inthe storage unit 4000 and read the various data from the storage unit4000. The function of the store/read processing unit 49 may beimplemented by the HDD 205 shown in FIG. 3.

The transmitting/receiving unit 41 in the external input device 40according to the embodiment is an example of a resolution informationreceiving unit, the display data transmitting unit 452 is an example ofan display data transmitting unit, the resolution converting unit 453 isan example of a resolution conversion unit, and the display control unit47 is an example of a display control unit.

[Resolution Management Table]

The storage unit 4000 includes a resolution management DB 4001 formed ofa resolution management table 4001T shown in FIG. 14. In the resolutionmanagement table 4001T, a resolution of the vertical direction (heightdirection) of the display 120 of the transmission terminal 10 and aresolution of the horizontal direction (width direction) which areindicated by the resolution information received from the transmissionterminal 10 at the transmitting/receiving unit 41 are associatedtogether and stored. For example, in the resolution management table4001T shown in FIG. 14, the resolution (width direction) and theresolution (height direction) of the display 120 of the terminal 10 are1024 and 768, respectively.

[Process/Operation of Embodiment]

Next, processes performed by the transmission system 1 according to theembodiment will be described with reference to FIGS. 15-20. Note thatFIG. 15 is a sequence diagram for explaining a process to manage stateinformation sent from relay devices 30 to the transmission managementsystem 50, the state information indicating respective operating statesof the relay devices 30. FIG. 16 is a sequence diagram for explaining apreparatory process performed to start communication between thetransmission terminals 10. FIG. 17 is a sequence diagram for explaininga process performed by a transmission terminal to establish a sessionwith another transmission terminal. FIG. 18 is a sequence diagram forexplaining a process performed to transmit display data from an externalinput device 40 of the transmission terminal to another transmissionterminal of a communication partner and display an image of the displaydata thereon. FIG. 19 is a diagram showing an example of display datadisplayed on the display 120. FIG. 20 is a diagram showing a screendisplayed on the display 216 aa of the external input device 40 aa andscreens displayed on the transmission terminals 10 aa and 10 db in thetransmission system 1.

With reference to FIG. 15, the process to manage the state informationtransmitted from the relay devices 30 to the management system 50 andindicating the operating states of the relay devices 30 will bedescribed.

Each of the relay devices 30 (30 a-30 d) includes the state detectingunit 32 (FIG. 4B) which regularly detects the operating state of therelay device 30 (itself) (S1-1 to S1-4). The transmitting/receivingunits 31 of the relay devices 30 regularly transmit a correspondingstate information of the relay device itself to the management system 50via the communication network 2 so that the management system 50 sidemay manage the operating states of the relay devices 30 in real time(step S2-1 to S2-4). The state information includes the relay device IDof the relay device 30 itself and the operating state detected by thestate detecting unit 32 of the relay device 30 itself and associates itwith the relay device ID. Note that in this embodiment, a case where therelay devices (30 a, 30 b, 30 d) are running normally and in the state“ONLINE” is illustrated; however, the relay device 30 c is operating butis in the state “OFFLINE” due to some kind of defect in the programs forexecuting the operation of the relay device 30 c.

Next, in the management system 50, the transmitting/receiving unit 51receives the state information sent from each of the relay devices 30a-30 d and the store/read processing unit 59 stores each of the receivedstate information in association with the corresponding relay device IDin the relay device management DB 5001 (the relay device managementtable 5001T of FIG. 7) of the storage unit 5000 (steps S3-1 to S3-4).Thus, the operating state “ONLINE”, “OFFLINE” or “FAILED” is stored inthe operating-state field of the relay device management table 5001T(FIG. 7) in association with the relay device ID for each of the relaydevices 30. Simultaneously, the date/time of reception of the stateinformation received by the management system 50 is also stored inassociation with the corresponding relay device ID in the relay devicemanagement table. Note that when the state information is not sent fromthe relay device 30, the operating state field and the receipt date/timefield of the record in the relay device management table (FIG. 7) may beblank, or the previous operating state and the previous receiptdate/time may be present.

Next, a preparatory process performed before starting communicationbetween the terminal 10 aa and the terminal 10 db will be described withreference to FIG. 16.

First, when the user of the terminal 10 aa switches ON the power switch109 (FIG. 2), the operation input receiving unit 12 (FIG. 4B) receives apower-ON signal to switch ON the power of the terminal 10 aa (step S21).

On receiving the power-ON signal as a trigger, the login request unit 13automatically transmits the login request information indicating thelogin request from the transmitting/receiving unit 11 to the managementsystem 50 via the communication network 2 (step S22). This login requestinformation includes a terminal ID and a password for identifying theterminal 10 aa indicating its own terminal as the request sourceterminal. The terminal ID and the password are data that are read fromthe storage unit 1000 via the store/read processing unit 19 and thensent to the transmitting/receiving unit 11. Note that when the loginrequest information is transmitted from the terminal 10 aa to themanagement system 50, the receiver side management system 50 may detectthe IP address of the transmitter side terminal 10 aa.

Subsequently, the terminal authenticating unit 52 (FIG. 4A) of themanagement system 50 authenticates the terminal 10 aa based on whetherthe terminal ID and the password contained in the login requestinformation received via the transmitting/receiving unit 51 areidentical to those managed in the terminal authentication management DB5002 (step S23). In this case, the terminal authenticating unit 52acquires the terminal ID and the password from the terminalauthentication management DB 5002 of the storage unit 5000 by searchingfor the terminal ID and the password as search keys in the terminalauthentication management table 5002T (FIG. 8).

When it is determined by the terminal authenticating unit 52 that theterminal ID and the password contained in the login request areidentical to those managed in the terminal authentication management DB5002 and the received login request comes from the terminal 10 having avalid access authorization, the state managing unit 53 stores theterminal ID of the terminal 10 aa, the operating state, the date/time ofreception of the login request information, and the IP address of theterminal 10 aa in association with each other in the terminal managementDB 5003 (the terminal management table 5003T of FIG. 9) (step S24).Thus, the operating state “ONLINE”, the reception date/time“2009.11.10.13:40”, and the terminal IP address “1.2.1.3” are managed inassociation with the terminal ID “01 aa” in the terminal managementtable (FIG. 9).

Subsequently, the transmitting/receiving unit 51 of the managementsystem 50 transmits authentication result information indicatingauthentication result obtained by the terminal authenticating unit 52via the communication network 2 to the request source terminal 10 aathat has sent the login request (step S25). In the following, the casewhere the terminal authenticating unit 52 has determined that theterminal 10 aa has the valid access authorization will be furtherdescribed.

The terminal extracting unit 54 of the management system 50 searches thedestination list management table (FIG. 10) by the terminal ID “01 aa”of the request source terminal 10 aa as a search key, and extracts theterminal IDs of the destination candidate terminals 10 capable ofcommunicating with the request source terminal 10 aa (step S26). As aresult, in this example, the terminal IDs “01 ab”, “01 ba”, “01 db” ofthe destination candidate terminals 10 ab, 10 ba, 10 db associated withthe terminal ID “01 aa” of the request source terminal 10 aa areextracted.

Subsequently, the terminal state acquiring unit 55 searches the terminalmanagement table 5003T (FIG. 9) by the terminal IDs (“01 ab”, “01 ba”,“01 db”) of the destination candidate terminals 10 extracted by theterminal extracting unit 54 as search keys, and retrieves the operatingstates of the destination candidate terminals (10 ab, 10 ba, 10 db) byreading the operating states (“OFFLINE”, “ONLINE”, “ONLINE”)corresponding to the terminal IDs extracted by the terminal extractingunit 54 (step S27).

Subsequently, the transmitting/receiving unit 51 transmits destinationstate information containing the terminal IDs (“01 ab”, “01 ba”, “01db”) used as the search keys at step S27 and the operating states(“OFFLINE”, “ONLINE”, “ONLINE”) of the corresponding destinationcandidate terminals (10 ab, 10 ba, 10 db) to the request source terminal10 aa via the communication network 2 (step S28). Thus, the requestsource terminal 10 aa may detect each of the current operating states(“OFFLINE”, “ONLINE”, “ONLINE”) of the destination candidate terminals(10 ab, 10 ba, 10 db) which are the candidate terminals for thedestination terminal capable of communicating with the request sourceterminal 10 aa.

Subsequently, the terminal extracting unit 54 of the management system50 searches the destination list management table 5004T (FIG. 10) by theterminal ID “01 aa” of the request source terminal 10 aa that has sentthe login request as a search key, and extracts terminal IDs of otherrequest source terminals 10 which have been registered as thedestination terminals 10 for the terminal ID “01 aa” of the requestsource terminal 10 aa (step S29). In the destination list managementtable 5004T shown in FIG. 10, the extracted terminal IDs of the otherrequest source terminals 10 are “01 ab”, “01 ba”, and “01 db”.

Subsequently, the terminal state acquiring unit 55 of the managementsystem 50 searches the terminal management table 5003T (FIG. 9) by theterminal ID “01 aa” of the request source terminal 10 aa that has sentthe login request as a search key, and acquires the operating state ofthe request source terminal 10 aa (step S30).

Subsequently, the transmitting/receiving unit 51 transmits destinationstate information containing the terminal ID “01 aa” and the operatingstate “ONLINE” of the request source terminal 10 aa acquired at step S30to the terminals (10 ba, 10 db) the operating states of which arepresented as “ONLINE” in the terminal management table 5003T (FIG. 9)among the terminals (10 ab, 10 ba, 10 db) corresponding to the terminalIDs (“01 ab”, “01 ba”, “01 db”) extracted at step S29 (steps S31-1 andS31-2). Note that, when the transmitting/receiving unit 51 transmits thestate information to the terminals (10 ba, 10 db), thetransmitting/receiving unit 51 refers to the IP addresses of theterminals managed in the terminal management table 5003T (FIG. 9) basedon the terminal IDs (“01 ba”, “01 db”). Thus, the terminal ID “01 aa”and the operating state “ONLINE” of the request source terminal 10 aamay be transmitted to each of the other destination terminals (10 db, 10ba) as the terminals capable of communicating with the request sourceterminal 10 aa that has sent the login request.

In a similar manner as step S21, when a user of another terminal 10switches ON the power switch 109 (FIG. 2), the operation input receivingunit 12 (FIG. 4B) receives a power-ON signal to switch ON the power ofthe other terminal 10, and the processes similar to the steps S22through S31-1, 31-2 are subsequently performed. Thus, the description ofthe steps S22 through S31-1, 31-2 is omitted.

Next, with reference to FIG. 17, a process in which a transmissionterminal establishes a session with another transmission terminal willbe described. In this embodiment, the request source terminal 10 aa iscapable of communicating with at least one of the terminals 10 ba and 10db the operating states of which are presented as “ONLINE” among thedestination candidate terminals (10 ab, 10 ba, 10 db) indicated by thedestination state information received at step S28. In the following, acase where the user of the request source terminal 10 aa selects thedestination terminal 10 db to initiate communication will be described.

Initially, when the user of the request source terminal 10 aa pressesthe operation button 108 (FIG. 2) to select the terminal 10 db, theoperation input receiving unit 12 (FIG. 4B) receives the selection ofthe terminal 10 db as the destination terminal (step S41). Next, thetransmitting/receiving unit 11 of the terminal 10 aa transmits startrequest information containing the terminal ID “01 aa” of the requestsource terminal 10 aa and the terminal ID “01 db” of the destinationterminal 10 db and indicating the request for starting communication tothe management system 50 (step S42). Thus, the transmitting/receivingunit 51 of the transmission management system 50 receives the startrequest information and detects the IP address “1.2.1.3” of the requestsource terminal 10 aa that is a transmission source terminal.

Based on the terminal ID “01 aa” of the request source terminal 10 aaand the terminal ID “01 db” of the destination terminal 10 db containedin the received start request information, the state managing unit 53changes the operating state fields of the records associated with theterminal IDs “01 aa” and “01 db” in the terminal management table 5003T(FIG. 9) of the terminal management DB 5003 to “BUSY”, respectively(step S43). In this state, the request source terminal 10 aa and thedestination terminal 10 db have not started communication but theoperating states of these terminals are set to a busy state, and ifother terminals 10 attempt to communicate with the request sourceterminal 10 aa or the destination terminal 10 db, a display or voicemessage indicating that the called terminal is in a busy state will beoutput.

The management system 50 generates a session ID used for execution ofthe session for selecting the relay device 30 (step S44). The sessionmanaging unit 57 stores the session ID “sel” generated at step S44, theterminal ID “01 aa” of the request source terminal 10 aa, and theterminal ID “01 db” of the destination terminal 10 db in the sessionmanagement table 5005T (FIG. 11) of the session management DB 5005 inassociation with each other (step S45).

Although, after the step S45, the management system 50 performsprocessing of focusing the relay device 30 for relaying communicationbetween the request source terminal 10 aa and the destination terminal10 db based on the relay device management DB 5001 and the terminalmanagement DB 5003, the processing performed by the management system 50is not relevant to this embodiment and therefore the description thereofis omitted.

When the above processing is completed, the session managing unit 57 ofthe management system 50 stores the relay device ID “lila” of thefinally selected relay device 30 a in the relay device ID field of therecord including the session ID “sel” in the session management table5005T (FIG. 11) of the session management DB 5005 (step S67-1). Thetransmitting/receiving unit 51 transmits the IP address “1.3.2.4” of thedestination terminal 10 db and the relay device ID “111 a” to therequest source terminal 10 aa (step S67-21).

The transmitting/receiving unit 51 of the management system 50 transmitsrelay start request information indicating a request for starting therelaying operation to the relay device 30 a via the communicationnetwork 2 (step S68). The relay start request information contains theIP addresses (“1.2.1.3”, “1.3.2.4”) of the request source terminal 10 aaand the destination terminal 10 db, so that the relay device 30 a mayreceive the IP addresses of the request source terminal 10 aa and thedestination terminal 10 db for relaying the communication between theterminals 10 aa and 10 db. The relay device 30 a establishes a sessionfor communicating image data with three resolutions (low, medium andhigh resolutions) and voice data between the terminal 10 aa and theterminal 10 db (step S69). Thus, a videoconference between the terminals10 aa and 10 db may be started. The image data with the threeresolutions which are communicated between the terminals in thisembodiment are an example. Alternatively, image data with four or moreresolutions may be communicated between the terminals.

Each terminal 10 is capable of transmitting and receiving image data byusing known moving image coding standard specifications, such as H.264(H.264/AVC, MPEG4 part10, MPEG4 AVC), H.264/SVC, or MPEG2.

Next, a process performed, after the relay device 30 is determined, totransmit display data indicating an image displayed on the display 216of the external input device 40 to another terminal 10, and display animage of the display data on the display 120 of the other terminal 10will be described with reference to FIG. 18. In the following, a casewhere the display data indicating the image displayed on the externalinput device 40 aa connected to the terminal 10 aa are transmitted toand displayed on the destination terminal 10 db will be described.

After the relay device 30 is determined as described above, thetransmitting/receiving unit 11 of the terminal 10 aa receives the IPaddress “1.3.2.4” of the destination terminal 10 db and the relay deviceID “111 a” which are transmitted by the management system 50 at stepS67-21. The store/read processing unit 19 of the terminal 10 aa storesin the storage unit 1000 the relay device ID “111 a” and the IP address“1.3.2.4” thus received (step S67-22).

When the external input device 40 aa is connected to the terminal 10 aa,the connection detecting unit 42 (FIG. 13) of the external input device40 aa detects the connection (step S70). When operation of the externalinput device 40 aa is started, the display control unit 47 displays atleast a screen including a whole area on the display 216.

After the connection detecting unit 42 detects the connection of theexternal input device 40 aa and the terminal 10 aa, the installationdetermining unit 43 a determines whether the display data acquiring unit451, the display data transmitting unit 452, and the resolutionconverting unit 453 are installed in the external input device 40 aa(step S71). Specifically, the installation determining unit 43 adetermines whether the display data acquisition program 1451 (FIG. 4B),the display data transmission program 1452, and the resolutionconversion program 1453 are installed in the external input device 40aa. When each program is installed, it is determined that thecorresponding functional units are installed in the external inputdevice 40 aa.

On the other hand, when the installation determining unit 43 adetermines at step S71 that any of the display data acquiring unit 451,the display data transmitting unit 452 and the resolution convertingunit 453 are not installed in the external input device 40 aa, themounting unit 48 mounts the storage unit 1000 of the terminal 10 aa(step S72). Subsequently, the program acquiring unit 43 b acquires andinstalls the program corresponding to the functional unit not installedin the external input device 40 aa, from among the display dataacquisition program 1451, the display data transmission program 1452,and the resolution conversion program 1453, stored in the storage unit1000 of the terminal 10 aa (step S73). As a result, the display dataacquiring unit 451, the display data transmitting unit 452, and theresolution converting unit 453 are installed in the external inputdevice 40 aa. Alternatively, the display data acquisition program 1451,the display data transmission program 1452, and the resolutionconversion program 1453 may be installed beforehand in the externalinput device 40 aa.

After the processing of step S73 is completed, thetransmitting/receiving unit 41 transmits information for requestingpermission to execute the programs (i.e., permission to operate thedisplay data acquiring unit 451, the display data transmitting unit 452,and the resolution converting unit 453) to the terminal 10 aa (stepS74). After the transmitting/receiving unit 41 receives informationindicating the permission from the terminal 10 aa in response to therequest, the display data acquiring unit 451 acquires the display dataindicating an image currently displayed on the display 216 (step S75).

Subsequently, the resolution converting unit 453 converts a firstresolution of the display data acquired by the display data acquiringunit 451 into a second resolution, and the display data transmittingunit 452 transmits the display data with the second resolution to theexternal information transmitting/receiving unit 18 of the terminal 10aa (step S76). The process performed by the resolution converting unit453 to convert the resolution of the display data will be describedlater. The process performed by the resolution converting unit 453 toconvert the resolution of the display data differs from the processperformed based on the operating state of the relay device 30 and thetransmission speed between the terminal 10 and the relay device 30, andthe conversion of the resolution of the display data does not depend onthe operating state of the relay device 30 and the transmission speedbetween the terminal 10 and the relay device 30.

After the external information transmitting/receiving unit 18 of therequest source terminal 10 aa receives the display data from theexternal input device 40 aa, the store/read processing unit 19 reads theIP address “1.3.2.4” of the destination terminal 10 db and the relaydevice ID “111 a” from the storage unit 1000 (step S77). Thetransmitting/receiving unit 11 transmits the display data with theconverted resolution and the IP address “1.3.2.4” of the destinationterminal 10 db to the relay device 30 indicated by the relay device ID“111 a” obtained at step S74 (step S78). After the display data and theIP address “1.3.2.4” sent by the terminal 10 aa at step S78 arereceived, the relay device 30 changes the image quality of the displaydata based on the IP address “1.3.2.4” (step S79), and transmits thedisplay data to the destination terminal 10 db indicated by the IPaddress “1.3.2.4” (step S80).

After the transmitting/receiving unit 11 of the destination terminal 10db receives the display data from the relay device 30, the displaycontrol unit 14 b displays the image indicated by the received displaydata on the display 120 (step S81).

FIG. 19 shows an example of the display data on the display 120 of thedestination terminal 10 db. In the example shown in FIG. 19, an image ofdisplay data D40 aa on the display 216 of the external input device 40aa is displayed at a left portion of the display screen on the display120, an image of display data D10 aa obtained by the imaging unit 14 aof the terminal 10 aa and transmitted by the transmitting/receiving unit11 is displayed at an upper right portion of the display screen on thedisplay 120. Further, an image of display data D10 db obtained by theimaging unit 14 a of the terminal 10 db is displayed at a lower rightportion of the display screen on the display 120.

FIG. 20 is a diagram showing an example of a screen displayed on thedisplay 216 aa of the external input device 40 aa, and screens displayedon the displays 120 aa, 120 db of the terminals 10 aa, 10 db in theoverall transmission system 1. On the display 216 aa of the externalinput device 40 aa, the screen which the user of the terminal 10 aadesires to share with the terminal 10 db is displayed. In the following,the screens displayed on the displays 216 aa, 216 db of the externalinput devices 40 aa, 40 db will be called desktop screens. The desktopscreen includes a whole area in which two or more areas may be arranged.In a certain case, a desktop screen including a whole area in which noarea is arranged may be displayed. In FIG. 20, the desktop screen inwhich the whole area, including the area A and the area B, is displayedis illustrated as an example.

By performing the process of FIG. 18, when the terminal 10 aa displaysthe image data and the display data on the display 120 aa, the terminal10 db displays the image data and the display data on the display 120db. In the following, such a screen will be called a terminal screen. Inthe terminal screen in the middle portion of FIG. 20, an image of thewhole area is displayed as the display data and images of users aa anddb of the terminals 10 aa and 10 db are displayed as the image data. Inthe terminal screen of the lowermost portion of FIG. 20, an image of thearea A is displayed as the display data and images of the users aa anddb of the terminals 10 aa and 10 db are displayed as the image data.

Note that the whole area and the specific areas which the user dbdisplays on the display 216 db of the external input device 40 db aredisplayed, and they are not affected by operation of the user aa. Inthis embodiment, the user may selectively display an arbitrary image ofthe whole area or the areas A and B on the display of the transmissionterminal 10.

Embodiment 1

In this embodiment, the transmission system 1 in which the image of thewhole area or arbitrary areas in the terminal screen may be selected anddisplayed when the user simply operates a pointing device (e.g., themouse 212) of the external input device 40 will be described. In thefollowing, the pointing device is implemented by the mouse 212.Alternatively, when a touch panel is utilized as the display 216, thepointing device may be implemented by a function to detect a position ofa user's finger on the touch panel.

FIG. 21A is a diagram showing an example of a desktop screen currentlydisplayed on the display 216 aa of the external input device 40 aa. Theprograms for the external input devices are installed in the externalinput device 40 aa. For example, a share setting input button 301 isdisplayed at the lower right corner of the desktop screen by apredetermined operation performed by the user. For example, thepredetermined operation may be to click or double-click an icon of theprograms for the external input devices by the mouse, to place the mouseover the icon, etc.

FIGS. 21B and 21C are diagrams showing examples of the share settinginput button 301. Specifically, the share setting input button 301displayed with the characters “start sharing” is shown in FIG. 21B, andthe share setting input button 301 displayed with the characters “stopsharing” is shown in FIG. 21C.

If the user clicks the share setting input button 301 of FIG. 21B by thepointing device, sharing of the display data displayed on the display216 aa of the external input device 40 aa is started. In the state wherethe display data displayed on the display 216 aa of the external inputdevice 40 aa is shared, the share setting input button 301 of FIG. 21Cis displayed. If the user clicks the share setting input button 301 ofFIG. 21C by the pointing device, sharing the display data displayed onthe display 216 aa of the external input device 40 aa with the terminal10 db is stopped. In this case, the display data are not transmitted tothe terminal 10 aa.

The operation input receiving unit 46 (FIG. 13) receives the useroperation on the share setting input button 301. When the user clicksthe share setting input button 301 displayed with “start sharing”, thedisplay data acquiring unit 451 acquires the display data correspondingto the area selected by the user.

In the transmission system 1 of this embodiment, the display data of thewhole area or the specific area in the external input device 40 in whichthe share setting input button 301 displayed with “start sharing”clicked at the last are shared. Therefore, when the user db clicks theshare setting input button 301 displayed with “start sharing” of FIG.21B, sharing of the display data displayed on the display 216 of theexternal input device 40 aa is automatically stopped even if the user aadoes not click the share setting input button 301 with “stop sharing” ofFIG. 21C.

[Operation Processes]

FIG. 22 is a sequence diagram for explaining a process of resolutionconversion performed by the external input device 40 and resolutionchanging performed by the transmission terminal 10 when a non-compressedform is used. FIG. 23 is a flowchart for explaining a process in whichthe display data acquiring unit 451 acquires the whole area display dataor the specific area display data. FIG. 24 is a flowchart for explaininga process to convert a resolution. FIGS. 25A and 25B are a flowchart forexplaining a process performed by the terminal 10 to control theresolution control (including changing the resolution). FIGS. 26A and26B are diagrams for explaining a determination step S102 in the processof FIG. 25A. FIG. 27 is a sequence diagram for explaining a process ofresolution conversion by the external input device 40 and resolutionchanging by the terminal 10 when a compressed form is used.

Referring to the sequence diagram shown in FIG. 22, the process ofresolution conversion of the display data to be transmitted to theterminal 10 by the external input device 40 based on the resolution ofthe display 120 of the terminal 10 and the resolution of the display 216of the external input device 40. In the example of FIG. 22, a case wherethe display data are transmitted in a non-compressed form is described.

As shown in FIG. 22, first, the transmitting/receiving unit 41 of theexternal input device 40 transmits a request of resolution informationregarding the display 120 to the terminal 10 (step S901). The resolutionacquiring unit 21 a of the terminal 10 acquires the resolutioninformation indicating a resolution of the display 120 of the terminal10 stored in the resolution storing unit 1005 (step S902). After theresolution acquiring unit 21 a acquires the resolution information atstep S902, the external information transmitting/receiving unit 18transmits the acquired resolution information to the external inputdevice 40 (step S903).

In the external input device 40, after the transmitting/receiving unit41 receives the resolution information regarding the display 120 of theterminal 10, the store/read processing unit 49 stores the receivedresolution information in the resolution management table 4001T of theresolution management DB 4001 (step S904).

Note that, although the resolution information regarding the display 120of the terminal 10 is acquired at step S902, it is preferred to givepriority to a resolution according to a mode specified by a user andconsider the resolution as the resolution information regarding thedisplay 120. For example, the mode may include a current mode, a highmode (640×360), a medium mode (320×180), and a low mode (192×108). If itis detected that the transmission of the display data is overloaded, theuser may promote the transmission of the display data by changing themode suitably. Namely, when the bandwidth is narrow, the user may selectthe transmission of the display data with a lower resolution rather thanthe resolution of the display 120, so as to prevent the transmission ofthe display data from being overloaded.

Subsequently, the display data acquiring unit 451 of the external inputdevice 40 acquires the display data indicating an image which isdisplayed on the display 216 by the display control unit 47 (step S905).

Referring to FIG. 23, the process of step S905 in which the display dataacquiring unit 451 acquires the display data will be described. It isassumed that the share setting input button 301 shown in FIG. 21B isalready clicked by the user aa.

In the process shown in FIG. 23, first, the operation input receivingunit 46 detects whether selection of the whole area is received (S9051).To select the whole area, the user aa clicks a portion other than adesktop screen area by the pointing device. The OS (operating system)detects a position of the clicked portion. In this embodiment, the OSwhich is operating on the external input device 40 treats each of thespecific areas and the whole area as one area, and regards the clickingthe portion other than the desktop screen area as selection of the wholearea. Namely, the whole desktop screen is treated as one area (the wholearea and the specific areas are treated separately). In the OS, an API(application programming interface) which enables selection of the wholearea indicating the whole display screen (desktop screen) is defined.Therefore, when the user aa selects the whole area by the pointingdevice, the operation input receiving unit 46 may detect selection ofthe whole area through the API, so that the whole area may be shared asthe display data. An intuitive switching operation may be performed bythe user. According to a certain OS, the area may be referred to as anobject (processing object), and the OS manages a position of the objectand an attribute value of the object and detects whether the object isan operation target. Such areas and objects are equivalent to programcomponents.

When the selection of the whole area is received (YES in S9051), thedisplay data acquiring unit 451 acquires the display data of the wholedesktop screen displayed on the display 216 (S9052), and the process ofFIG. 23 is terminated.

When the selection of the whole area is not received (NO in S9051), thedisplay data acquiring unit 451 determines whether an active area ispresent (S9053). For example, the function of the OS is used for thisdetermination. In a case where the Windows® API is utilized, a handle ofan active area is acquired, and if the value of the handle is not NULL,it is determined that an active area is present. The active arearepresents an area which is input by a user or currently set to an,operation target. In the case of Windows® or Mac® OS, the active area iscalled an active window.

When the user aa closes the active area and there is no active area (NOin S9053), the process progresses to step S9052. In step S9052, thedisplay data acquiring unit 451 acquires the display data of the wholedesktop screen. When the active area is present (YES in S9053), thedisplay data acquiring unit 451 acquires the display data of the areawhich is active among the display data displayed on the display 216 bythe display control unit (S9054). This area is specified by the handleof the active area acquired at step S9053, and the display dataacquiring unit 451 may retrieve the display data of the area to beacquired.

Next, the display data acquiring unit 451 determines whether a width ora height of the area of the acquired display date is smaller than athreshold (S9055). It is assumed that the threshold for the width isequal to 128 pixels and the threshold for the height is equal to 36pixels. This determination is made to prevent the processing load of ascaling process for resolution conversion from being increased. Thedisplay data of an area too small does not have meaningful content. Bydetermining the minimum size of the area for scaling, the load of theCPU when the rendering of the display data is performed on the terminal10 aa may be reduced.

When the width or the height of the active area is smaller than thethreshold (YES in S9055), black pixels are added to the display data bythe display data acquiring unit 451 until the size of the currentdisplay data is equal to the threshold width×the threshold height (e.g.,128×36 pixels) (S9056), and the process of FIG. 23 is terminated.

The black pixels are arranged as follows. First, an area having a sizeof the threshold width by the threshold height is prepared, and thecurrent display data are arranged so that the position of the upper leftcorner of the current display data is in agreement with the position ofthe upper left corner of the prepared area. The pixels within the areaof the threshold width by the threshold height in which the currentdisplay data are not arranged are replaced with black pixels.Alternatively, the current display data may be arranged at any of otherthree corners (the lower left corner, the upper right corner, the lowerright corner), or in the center position of the area of the thresholdwidth by the threshold height. The pixels to be added may not be blackpixels. Pixels of a different color may be added instead of blackpixels.

Referring back to FIG. 22, the resolution acquiring unit 44 acquires theresolution information of the display 216 (step S906). The resolutionconverting unit 453 determines the resolution of the display data to betransmitted to the terminal 10, based on the resolution which is storedin the resolution management DB 4001 and indicated by the resolutioninformation of the display 120 of the terminal 10 and the resolutionwhich is indicated by the resolution information of the display 216 ofthe external input device 40 and acquired at step S906 (step S907).

Next, the resolution conversion process performed by the resolutionconverting unit 453 at step S907 will be described with reference toFIG. 24.

It is assumed that a resolution of a vertical direction (height (H)direction) and a resolution of a horizontal direction (width (W)direction) included in the resolution information of the display 216 ofthe external input device 40 acquired at step S906 are set to “H40” and“W40”, respectively. Similarly, it is assumed that a resolution of thevertical direction (height (H) direction) and a resolution of thehorizontal direction (width (W) direction) included in the resolutioninformation of the display 120 of the terminal 10 stored in theresolution management DB 4001 are set to “H10” and “W10”, respectively.Based on these resolutions H40, W40, H10 and W10, the resolutionconverting unit 453 determines a resolution “H′” of the verticaldirection of the display data and a resolution “W′” of the horizontaldirection of the display data.

In the process of FIG. 24, first, the resolution converting unit 453determines whether the condition of (H40≦H10 and W40≦W10) is met (stepS9071). When it is determined that the condition of (H40≦H10 andW40≦W10) is met (YES in step S9071), the resolution converting unit 453determines the resolution H′ of the vertical direction and theresolution W′ of the horizontal direction of the display data as H′=H40and W′=W40, respectively (step S9072). In this case, the resolution ofthe display data remains unchanged.

On the other hand, when the condition of (H40≦H10 and W40≦W10) is notmet (NO in step S9071), the resolution converting unit 453 determineswhether the condition of (W40>W10 and H40≦H10) or (W40>W10, H40>H10 andH10≧W10) is met (step S9073). When the condition of (W40>W10 andH40≦H10) or (W40>W10, H40>H10 and H10≧W10) is met (YES in step S9073),the resolution converting unit 453 determines the resolution W′ of thehorizontal direction and the resolution H′ of the vertical direction ofthe display data as W′=W10, H′=H40*W10/W40, respectively (step S9074).W10/W40 represents a compression ratio of the width (W) direction.

On the other hand, when it is determined that the condition of (W40>W10and H40≦H10) or (W40>W10, H40>H10, and H10≧W10) is not met (NO in stepS9073), the resolution converting unit 453 determines the resolution W′of the horizontal direction and the resolution H′ of the verticaldirection of the display data as W′=W40*H10/H40 and H′=H10, respectively(step S9075). H10/H40 represents a compression ratio of the height (H)direction.

For example, in the case where it is determined at step S9073 that thecondition of (W40>W10, H40>H10 and H10≧W10) is not met, the resolutionof the vertical direction of the display 216 of the external inputdevice 40 is greater than the resolution of the vertical direction ofthe display 120 of the terminal 10, and the resolution of the horizontaldirection of the display 216 of the external input device 40 is equal toor smaller than the resolution of the horizontal direction of thedisplay 120 of the terminal 10 (H40>H10 and W40≦W10). Or in the casewhere it is determined at step S9073 that the condition of (W40>W10,H40>H10 and H10≧W10) is not met, the resolution of the horizontaldirection and the resolution of the vertical direction of the display216 of the external input device 40 are greater than the resolution ofthe horizontal direction and the resolution of the vertical direction ofthe display 120 of the terminal 10, respectively, and the resolution ofthe vertical direction of the display 120 of the terminal 10 is smallerthan the resolution of the horizontal direction thereof (W40>W10,H40>H10 and H10<W10).

Namely, when the resolution of the vertical direction of display 216 ofthe external input device 40 is smaller than the resolution of thevertical direction of the display 120 of the terminal 10, the resolutionconverting unit 453 at step S907 converts the resolution of the verticaldirection of the display data into the resolution of the verticaldirection of the display 120 of the terminal 10 (H′=H10). Further, theresolution converting unit 453 at step S907 converts the resolution ofthe horizontal direction of the display data such that an aspect ratioof the display data is equal to an aspect ratio of the resolution of thedisplay 120 of the terminal 10 (W=W40*H10/H40).

When the resolution of the horizontal direction of the display 120 ofthe terminal 10 is smaller than the resolution of the horizontaldirection of the display 216 of the external input device 40, theresolution converting unit 453 at step S907 converts the resolution ofthe horizontal direction of the display data into the resolution of thehorizontal direction of the display 120 of the terminal 10 (W′=W10).Further, the resolution converting unit 453 at step S907 converts theresolution of the vertical direction of the display data such that anaspect ratio of the display data is equal to an aspect ratio of thedisplay 120 of the terminal 10 (H′=H40*W10/W40).

Referring back to FIG. 22, after the resolution of the display data isconverted by the resolution converting unit 453, the display datatransmitting unit 452 transmits the display data with the convertedresolution to the terminal 10 (step S908).

After the external information transmitting/receiving unit 18 of theterminal 10 receives the display data from the external input device(step S908), the display control unit 14 b performs the rendering of thereceived display data and displays an image of the display data on thedisplay 120 (step S909). The display image acquiring unit 14 c acquiresthe data which indicate the image displayed on the display 120 by thedisplay control unit 14 b at step S909 (step S910). Thetransmitting/receiving unit 11 transmits the image data acquired by theimage acquiring unit 14 c to the relay device 30 (step S911).

When the display data with the unchanged resolution is received (in thecase of S9072), the display control unit 47 performs the rendering ofthe received display data by changing the received resolution to theresolution used for the display data on the display 120, with the ratioof the width to the height of the display data maintained.

For example, when the full screen displaying of the display data isperformed with the resolution of the display 120, the rendering of thedisplay data is performed as follows.

Wr=Wt, Hr=Hs*Wt/Ws (if Wt/Ws<Ht/Hs)

Hr=Ws*Ht/Hs, Hr=Ht (else)

where Wr and Hr represent the width and the height after the renderingrespectively, Ws and Hs represent the width and the height of thereceived display data respectively, and Wt and Ht the width and theheight of the display respectively.

After the operation input receiving unit 12 receives the resolutioninformation, the resolution changing unit 21 d changes the resolution ofthe display 120 of the terminal 10 (step S912). This process ofresolution changing is performed by a process of resolution changing ofsteps S105 S109, S117, S122 in the process of FIGS. 25A and 25B.

After the resolution is changed, the resolution acquiring unit 21 aacquires the resolution information indicating the changed resolution(step S902). The external information transmitting/receiving unit 18transmits the resolution information to the external input device 40(step S903). After this, the process which is the same as the processperformed at the above step S904 and subsequent steps is performed, andthe explanation thereof is omitted.

Next, with reference to FIGS. 25A and 25B, the process of resolutioncontrol (including resolution changing) performed by the terminal 10will be described.

As shown in FIG. 25A, when the process is started by reception of apower-ON signal in the terminal 10, the resolution acquiring unit 21 a(FIG. 4B) accesses the display 120 to acquire available resolutionswhich can be displayed on the display 120 (step S101).

Subsequently, the resolution determining unit 21 b determines whetherresolutions other than a standard aspect ratio (e.g., 16:9, and itscorresponding resolution: 1280×720) used for communication of the imagedata between the terminal 10 and the relay device 30 are included in theacquired resolutions which can be displayed on the display 120 (stepS102). This determination is made because there may be a case where theuser desires to select a resolution other than the standard aspectratio, if possible, and add a segment area for presenting supplementaryinformation. FIG. 26A shows an example of a screen in such a case. Inthe example shown in FIG. 26A, a segment area SR is added to the bottomof the screen, and supplementary information AX (including an errormessage and an operation button) is displayed in the segment area SR.

On the other hand, FIG. 26B shows another example. In the example shownin FIG. 26B, a segment area is not added but the supplementaryinformation AX (including the error message and the operation button) isinserted in a bottom portion of the screen, and the screen with theinserted supplementary information AX is displayed.

When the resolutions other than the standard aspect ratio are included(YES in step S102), the resolution selecting unit 21 c selects aresolution from among the resolutions other than the standard aspectratio according to predetermined criteria (step S103). The predeterminedcriteria used at step S103 may select the highest resolution (or withthe maximum width and the maximum height), to select a resolutioncorresponding to a specific aspect ratio (e.g., 8:5), to select aresolution suitable for a character size, the number of characters, etc.of the supplementary information, or to cause a user to define aspecific resolution (e.g., WXGA (1280×800) or XGA (1024×768)) to beselected by the resolution selecting unit 21 c. Any of the criteria maybe used.

Subsequently, an image processing unit (not shown) which is provided asa function of the display control unit 14 b adds a segment area andexpands or contracts the image data internally generated at the standardaspect ratio to meet the selected resolution (step S104). Specifically,the image processing unit performs the following process. First, theimage data before conversion are expanded or contracted symmetrically tobe in agreement with the width after conversion. When the width beforeconversion is the same the width after conversion, expansion orcontraction of the image data is not performed (expansion ratio orcontraction ratio is set to 1). Next, a height of the image data afterexpansion or contraction is checked and a segment area is added to theimage data after the expansion or contraction to meet the height afterthe expansion or contraction. In the example shown in FIG. 26A, thesegment area SR is added to the bottom of the screen. Note that thelocation to which the segment area is added is not limited to the bottomof the screen. Alternatively, the segment area may be added to a topportion of the screen, or segment areas with a half height may be addedto the top and bottom of the screen, respectively.

Subsequently, the resolution changing unit 21 d changes the resolutionof the display 120 to the selected resolution (step S105). As a result,the image of the display data is displayed on the display 120 with thechanged resolution.

Subsequently, a flag setting unit (not shown) which is a function of thedisplay control unit 14 b sets the flag “overlap” to the value “false”(step S106). This flag “overlap” is accessed when the display controlunit 14 b detects an error and performs display control of supplementaryinformation different from the currently displayed image data. The flag“overlap” is set to the value “true” when the supplementary informationoverlapping the image data is displayed (the case of FIG. 26B), and setto the value “false” when the supplementary information not overlappingthe image data is displayed (the case of FIG. 26A).

On the other hand, when the resolutions other than the standard aspectratio are not included (NO in step S102), the resolution selecting unit21 c selects a resolution of the standard aspect ratio according topredetermined criteria (step S107). The predetermined criteria are thesame as those used at step S103.

Subsequently, an image processing unit (not shown) which is a functionof the display control unit 14 b expands or contracts the image datainternally generated at the standard aspect ratio to meet the selectedresolution (step S108). In this case, expansion or contraction of theimage data at the same aspect ratio is performed, a blank area is notgenerated in both the width direction and the height direction of theimage data, and a segment area is not added (see the example of FIG.26B).

Subsequently, the resolution changing unit 21 d changes the resolutionof the display 120 to the selected resolution (step S109). As a result,the image of the display data is displayed on the display 120 with thechanged resolution.

Subsequently, the flag setting unit (not shown) which is the function ofthe display control unit 14 b sets the flag “overlap” to the value“true” (step S110).

The foregoing process shown in FIG. 25A is an initialization processperformed upon receiving the power-ON signal in the terminal 10. Afterthis, a loop process shown in FIG. 25B may be performed when the display120 is exchanged after the reception of the power-ON signal.

As shown in FIG. 25B, the resolution acquiring unit 21 a accesses thedisplay 120 to acquire the currently used resolution of the display 120(step S111). Subsequently, the resolution acquiring unit 21 a accessesthe display 120 to acquire the available resolutions which can bedisplayed on the display 120 (step S112).

Subsequently, the resolution determining unit 21 b determines whetherresolutions other than the standard aspect ratio (e.g., 16:9) used forcommunication of the image data between the terminal 10 and the relaydevice 30 are included in the acquired resolutions (step S113). When theresolutions other than the standard aspect ratio are included (YES instep S113), the resolution selecting unit 21 c selects a resolution fromamong the resolutions other than the standard aspect ratio according tothe predetermined criteria (step S114). The predetermined criteria arethe same as those used at step S103.

Subsequently, the resolution determining unit 21 b determines whetherthe currently used resolution is different from the selected resolution(step S115). When the currently used resolution is different from theselected resolution (YES in step S115), the image processing unit (notshown) which is a function of the display control unit 14 b adds asegment area and expands or contracts the image data internallygenerated at the standard aspect ratio to meet the selected resolution(step S116).

Subsequently, the resolution changing unit 21 d changes the resolutionof the display 120 to the selected resolution (step S117). As a result,the image of the display data is displayed on the display 120 with thechanged resolution.

Subsequently, the flag setting unit (not shown) which is a function ofthe display control unit 14 b sets the flag “overlap” to the value“false” (step S118). On the other hand, when the currently usedresolution is the same as the selected resolution (NO in step S115), theprocessing of steps S116-S118 is not performed. This is because theresolution changing is not required.

On the other hand, when the resolutions other than the standard aspectratio are not included (NO in step S113), the resolution selecting unit21 c selects a resolution of the standard aspect ratio according to thepredetermined criteria (step S119). The predetermined criteria are thesame as those used at step S103.

Subsequently, the resolution determining unit 21 b determines whetherthe currently used resolution is different from the selected resolution(step S120). When the currently used resolution is different from theselected resolution (YES in step S120), the image processing unit (notshown) which is a function of the display control unit 14 b expands orcontracts the image data internally generated at the standard aspectratio to meet the selected resolution (step S121).

Subsequently, the resolution changing unit 21 d changes the resolutionof the display 120 to the selected resolution (step S122). As a result,the image of the display data is displayed on the display 120 with thechanged resolution.

Subsequently, the flag setting unit (not shown) which is a function ofthe display control unit 14 b sets the flag “overlap” to the value“true” (step S123).

On the other hand, when the currently used resolution is the same as theselected resolution (NO in step S120), the processing of steps S121-S123is not performed. This is because the resolution changing is notrequired.

The foregoing process is repeatedly performed while the program isoperating. Therefore, even if the display 120 is exchanged with anotherdisplay 120 with a different resolution, the process of FIG. 25B iscontinuously performed. The repeated process is equivalent to a processto regularly monitor a change of the resolution of the display 120. Thisprocess may be performed by using the known event handling program inthe programs of Windows®. In this case, the program may be registeredupon starting the process, such that an event notification may beperformed when a change of the resolution of the display 120 is detecteddue to the connection of the terminal 10 to another display 120.Thereafter, each time the notification of this event takes place, theprocess starting from step S111 of FIG. 25B may be performed by thedisplay control unit 14 b.

[Modification of Embodiment 1]

In the embodiment of FIG. 22, the case where the external input device40 transmits the display data in a non-compressed form (a bitmap form)to the terminal 10 and the terminal 10 further transmits the displaydata in the non-compressed form to the relay device 30 has beendescribed. The invention is not limited to the transmission of thedisplay data in the non-compressed form. Alternatively, the externalinput device 40 may be configured to transmit the display data in acompressed form, such as JPEG or GIF.

In the following, a process performed in a case of using a compressedform will be described with reference to FIG. 27. The flowchart shown inFIG. 27 differ from the flowchart shown in FIG. 22 in that processing(steps S907-A, S908-A, S910-A) related to compression of display dataand decompression of the display data is added. Other steps in theprocess of FIG. 27 different from the newly added steps are essentiallythe same as corresponding steps in the process of FIG. 22, and theexplanation thereof is omitted.

In the process of FIG. 27, after the resolution converting unit 453 ofthe external input device 40 performs resolution conversion of thedisplay data at step S907, the display data acquiring unit 451 performscompression (encoding) processing of the display data (step S907-A). Inthis embodiment, the display data acquiring unit 451 performs thecompression processing.

In the terminal 10, after the external informationtransmitting/receiving unit 18 receives the compressed display data withthe converted resolution from the external input device 40 at step S908,the display image acquiring unit 14 c performs the decompression(decoding) of the compressed display data received (step S908-A). Inthis embodiment, the display image acquiring unit 14 c performs thedecompression of the compressed display data.

Subsequently, after the display control unit 14 b performs the renderingof the display data after the decompression (decoding) and displays theimage of the display data on the display 120 at step S909, the displayimage acquiring unit 14 c acquires the display data indicating the imagecurrently displayed on the display 120 (step S910). The acquired displaydata (in the non-compressed form) are provided to each of the resolutionchanging unit 21 d and the display image acquiring unit 14 c. Thedisplay image acquiring unit 14 c performs predetermined compression(encoding) on the acquired display data (step S910-A). Thetransmitting/receiving unit 11 transmits the compressed display dataobtained by the display image acquiring unit 14 c to the relay device 30via the communication network 2 at step S911.

The compressed display data are received by another transmissionterminal 10 (FIG. 1) through the relay device 30. In the terminal 10which has received the compressed display data, the display control unit14 b performs the decompression (decoding) of the compressed displaydata, performs the rendering of the display data, and displays the imageof the display data on the display 120. Thus, the display data may beshared by the destination terminal 10 and the request source terminal10.

On the other hand, the display data acquired by the display imageacquiring unit 14 c at step S910 are provided to the resolution changingunit 21 d, and the resolution changing unit 21 d performs the resolutionchanging at step S912. After this, the process which is the same as theprocess shown in FIG. 25A is performed, and the explanation thereof isomitted.

As described above, the display data which are transformed in thecompressed form are transmitted from the external input device 40 to theterminals 10 through the relay device 30 (the source terminal 10->therelay device 30->the destination terminal 10), and the capacity of datatransmission can be reduced to be smaller than that in the case (FIG.22) where the display data in the non-compressed form (bitmap form) areused, and the communication load can be reduced.

As described above, in the transmission system according to theembodiment, if the whole area is selected by the mouse and the displaydata of the whole area can be shared with the terminal of acommunication partner of a videoconference, and if the whole screen isnot selected, the display data of the active area can be shared with theterminal of the communication partner of the videoconference. Therefore,one of the whole screen and some specific areas can be arbitrarilyselected and the display data of the selected area can be shared by theterminals.

Embodiment 2

In this embodiment, the transmission system 1 in which the image of thewhole area or arbitrary areas in the terminal screen may be selected anddisplayed when the user operates a viewing-area selection menu displayedon the display 216 of the external input device 40 will be described.

FIGS. 28A, 28B and 28C are diagrams showing an example of a desktopscreen displayed on the external input device 40 aa similar to FIGS.21A, 21B and 21C of Embodiment 1. FIG. 28A shows an example of thedesktop screen displayed on the display 216 aa of the external inputdevice 40 aa. In the example of FIG. 28A, a viewing-area selection menu310 (display data transmission permission menu) is displayed at a lowerright corner of the desktop screen.

FIGS. 28B and 28C show examples of the viewing-area selection menu 310.Specifically, the share setting input button 301 displayed with thecharacters “start sharing” is shown in FIG. 28B, and the share settinginput button 301 displayed with the characters “stop sharing” is shownin FIG. 28C. The function of the share setting input button 301 is thesame as that of Embodiment 1.

As shown in FIGS. 28B and 28C, two radio buttons 302 and 303 aredisplayed in addition to the share setting input button 301. The radiobutton 302 is to select the “whole area (full screen)”, and the radiobutton 303 is to select the “active area”. The radio buttons 302 and 303are provided for the user to select one of display data of the wholearea and display data of the active area. When the user selects one ofthe radio buttons 302 and 303, the operation input receiving unit 46receives the selection of the one of the radio buttons 302 and 303.

When the “start sharing” is selected by the share setting input button301 (or when the display data are shared), the shared display datadisplayed on the display 120 of the terminal 10 aa are alternatelyswitched between the display data of the whole area and the display dataof the active area each time one of the radio buttons 302 and 303selected by the user is changed to the other.

When the “start sharing” is not selected by the share setting inputbutton 301 (or when the display data are not shared), sharing of thedisplay data of the selected one of the radio buttons 302 and 303 isstarted if the user clicks the share setting input button 301 “startsharing”.

The operation processes of this embodiment are essentially the same asthose of FIGS. 22-30 of Embodiment 1 except for the process of thedisplay data acquiring unit of this embodiment partially differing fromthe process of FIG. 23 of Embodiment 1. FIG. 29 is a flowchart forexplaining the process in which the display data acquiring unit 451acquires display data.

In the process shown in FIG. 29, the operation input receiving unit 46detects whether the user has selected the whole area by the radio button302 (S9051-2).

When the user has selected the whole area by the radio button 302 (YESin S9051-2), the display data acquiring unit 451 acquires the displaydata of the whole desktop screen displayed on the display 216 as thedisplay data to be transmitted to the terminal 10 (S9052), and theprocess of FIG. 29 is terminated.

When the user has not selected the whole area by the radio button 302(NO in S9051-2) (i.e., when the user has selected an active area by theradio button 303), the display data acquiring unit 451 determineswhether the active area is present (S9053). The determination method maybe the same as that of Embodiment 1.

When the user aa closes the active area and there is no active area (NOin S9053), the process progresses to step S9052. In step S9052, thedisplay data acquiring unit 451 acquires the display data of the wholedesktop screen. When the active area is present (YES in S9053), thedisplay data acquiring unit 451 acquires the display data of the areawhich is active among the display data displayed on the display 216 bythe display control unit (S9054). Subsequent steps in the process ofFIG. 29 may be the same as corresponding steps of Embodiment 1.

According to this embodiment, even when the OS does not support treatingthe whole desktop screen as one area, the user is enabled to select thewhole area by using the viewing-area selection menu 310.

Embodiment 3

In Embodiments 1 and 2, when the display data acquiring unit 451acquires the display data of the whole area or the active area, there isa problem in that a mouse cursor is not displayed on the display 216.

This problem may arise in a case where a Print Screen button of thekeyboard is used in acquiring the display data. The mouse cursor is notincorporated even if the Print Screen button is pressed. For thisreason, it is preferred to configure the display data acquiring unit 451to incorporate the mouse cursor in the display data, when the mousecursor points to the whole area or the active area.

FIG. 30A is a block diagram showing the functional configuration of theexternal input device of this embodiment. The functional configurationof FIG. 30A is similar to that of FIG. 13. FIG. 30B is a block diagramshowing the functional configuration of the display data acquiring unit451 of this embodiment.

As shown in FIG. 30B, the display data acquiring unit 451 includes acursor information acquiring unit 4511, a screen information acquiringunit 4512, and a display data generating unit 4513. The screeninformation acquiring unit 4512 is configured to acquire the displaydata displayed on the display 216 of the external input device 40. Thecursor information acquiring unit 4511 acquires cursor informationincluding a position of the mouse cursor, an icon of the mouse cursor,etc. The cursor information may include coordinate data of the mousecursor in the screen and an icon of the mouse cursor. The coordinatedata of the mouse cursor may be acquired by querying the OS by apredetermined command. Similarly, the icon of the mouse cursor may beacquired from the OS. Alternatively, a known icon of the mouse cursormay be included beforehand as image data.

The display data generating unit 4513 is configured to incorporate themouse cursor icon in the display data. Because the external input device40 transmits the display data to the terminal 10, the user may detectthe position where the mouse cursor specifies the display data in thescreen.

The method for incorporating the mouse cursor icon differs for a casewhere the whole area is designated as the display data and a case wherethe active area is designated as the display data.

The case where the whole area is designated as the display data will bedescribed with reference to FIG. 31. FIG. 31 is a diagram showing anexample in which the mouse cursor icon is incorporated in the displaydata. As shown in FIG. 31, the display data indicate the whole screendisplayed on the display 216 aa of the external input device 40 aa. Themouse cursor may move in the whole desktop screen.

The cursor information acquiring unit 4511 acquires the mouse cursoricon and the coordinate data of the mouse cursor. The coordinate dataare expressed by an orthogonal coordinate system and indicate atwo-dimensional position of the mouse cursor on a pixel-by-pixel basiswith the origin at the upper left corner of the screen.

Therefore, the display data generating unit 4513 may incorporate themouse cursor in the display data by placing the mouse cursor pointer atthe position of the display data indicated by the coordinate data.Incorporating the mouse cursor may include placing the mouse cursor iconin the display data and generating the display data (in which the pixelvalues of the overlapping pixels are included).

The case where the active area is designated as the display data will bedescribed with reference to FIG. 32. FIG. 32 is a diagram showing anexample in which the mouse cursor icon is incorporated in the displaydata. As shown in FIG. 32, the coordinates (X, Y) of the mouse cursorare coordinates when the upper left corner of the desktop screen isregarded as the origin, and the active area is an area of the desktopscreen. For this reason, if the mouse cursor icon is incorporated in thedisplay data of the active area at coordinates (X, Y), the icon islocated at a position on the basis of the coordinates (X′, Y′) of theupper left corner of the active area, and the position of the mousecursor icon in the display data will change.

To avoid the problem, when the active area is designated as the displaydata, the cursor information acquiring unit 4511 is configured tocompensate the coordinate data of the mouse cursor as follows.

X=X−X′

Y=Y−Y′

After the compensation of the coordinates is performed in this manner,the relative position of the mouse cursor icon to the display data ismaintained and the mouse cursor icon may be suitably incorporated in thedisplay data.

FIG. 33 is a flowchart for explaining a process in which the displaydata acquiring unit 451 incorporates the mouse cursor icon in thedisplay data. The process of FIG. 33 is performed at step S905 in theprocess of FIG. 22 after the process of acquisition of the display datain Embodiment 1 or Embodiment 2 is performed. In FIG. 33, it is assumedthat the display data of the active area are selected.

In the process of FIG. 33, first, the cursor information acquiring unit4511 acquires the coordinate data of the mouse cursor (S905-11). Whenthe display data of the whole area are selected, the mouse cursor iconmay be incorporated in the display data after this step.

The cursor information acquiring unit 4511 determines whether a positionindicated by the coordinate data acquired at step S905-11 is included inthe active area (S905-12).

When the mouse cursor position is not included in the active area (NO inS905-12), it is not necessary to incorporate the mouse cursor icon. Theprocess of FIG. 33 is terminated without incorporating the mouse cursoricon in the display data of the active area by the display datagenerating unit 4513.

When the mouse cursor position is included in the active area (YES inS905-12), the display data generating unit 4513 acquires the mousecursor icon (S905-13).

Subsequently, the display data generating unit 4513 incorporates themouse cursor icon acquired at step S905-13 in the display data of theactive area (S905-14), and the process of FIG. 33 is terminated. Thedisplay data generating unit 4513 accesses the coordinate data (X, Y) ofthe mouse cursor pointer and the coordinate (X′, Y′) of the upper leftcorner of the active area. On the basis of the origin at the upper leftcorner of the display data in which only the image of the active area isincorporated, the mouse cursor icon is incorporated in the display dataat the position of ((X−X′), (Y−Y′)).

As described above, even in the case where the display data of theactive area are shared, the mouse cursor can be correctly displayed onthe display 120 of the terminal 10 db by changing the position of themouse cursor icon from that in the case where the display data of thewhole area are shared.

In the process of FIG. 33, the display data of the active area areselected. Alternatively, the compensation process may be performed whenthe display data of the active area are selected, and when the displaydata of the whole area are selected, the coordinates (X′, Y′) of theupper left corner of the active area may be set to X′=0 and Y′=0.

Embodiment 4

As described above, in Embodiment 1, the terminal 10 aa in which, whenthe user selects the specific area on the external input device 40 bythe mouse, the display data of the active area are shared with theterminal 10 db used by a communication partner of a videoconference hasbeen explained.

Similarly, in Embodiment 2, the terminal 10 aa in which, when the userselects the radio button 303 in the viewing-area selection menu 310 ofthe external input device 40, the display data of the active area areshared with the terminal 10 db used by the communication partner of thevideoconference has been explained.

Although the active area can be easily changed by operation of the user,there may be a case where an area which is not suitable as a sharedobject is designated as the active area.

FIGS. 34A, 34B and 34C are diagrams showing examples of areas (windows)which are not be shared in Windows®. FIG. 34A shows an area calledBaseBar 401 which is displayed when a start button is clicked. The usercan select arbitrary one of items in the displayed menu.

In Windows®, this area is also treated as a window. Therefore, theBaseBar area is designated as an active area. For this reason, thedisplay data of the BaseBar area are shared by the terminals 10 aa and10 db. However, the display data of the BaseBar area have littleinformation which is to be shared, and therefore the display data ofsuch an area should not be shared.

FIG. 34B shows an example of a thumbnail indication area 402 of a filemanagement application which is displayed when the icon of theapplication is pointed to by the mouse. In the example of FIG. 34B,three thumbnail screens generated by the file management application aredisplayed. The thumbnail indication area 402 in which the threethumbnails are displayed may become the active area. When the userselects one of the thumbnails, the screen generated by the filemanagement application is displayed on the display 216 as the activearea. Thus, the thumbnail indication area is recognized as the activearea when the user of the external input device 40 aa points to the iconof the task bar by the mouse. Although the thumbnail indication area isshared by the terminals 10 aa and 10 db, the thumbnail indication areahas little information which is to be shared, and therefore the displaydata of such an area should not be shared.

FIG. 34C shows an example of a pop-up area 403 which is generated by atool tip function. Similar to the thumbnail indication area of FIG. 34B,when the user points to the icon in which the tool tip function isembedded, the pop-up area 403 is displayed. The pop-up area may becomethe active area. Although the pop-up area is shared by the terminals 10aa and 10 db, the pop-up area has little information which is to beshared, and therefore the display data of such an area should not beshared.

In all the examples of FIGS. 34A-34C, the display data of active areasmay be shared against the user's intention.

In Embodiment 4, the external input device 40 aa in which, even if apredetermined area is designated as an active area, the display data ofthe predetermined area displayed on the display 216 aa are not acquiredwill be described. The external input device 40 aa does not acquire thedisplay data of the predetermined area even if the predetermined area isdesignated as the active area, and the terminals 10 aa and 10 db do notshare the display data.

FIG. 35 is a block diagram showing the functional configuration of theexternal input device 40, and FIG. 36 is a diagram showing an example ofan exception management table 4002T.

An exception management DB 4002 is implemented by the storage unit 4000and the exception management table 4002T as shown in FIG. 36 is storedin this exception management DB 4002. In the exception management table4002T, area names (window names) of predetermined areas the display dataof which are not acquired even when the predetermined areas aredesignated as the active area are registered. The exception managementtable 4002T is stored in the storage unit 1000 of the terminal 10 andinstalled in the external input device 40 together with a display dataacquisition program 1451.

When the Windows® API is utilized, an active area is notified as a classname from the OS. In addition, the area names of the areas shown inFIGS. 34A-34C are also uniquely determined by the OS. Thus, a softwaredeveloper may predetermine the areas the display data of which are notto be acquired even if the predetermined areas are designated as theactive are, and register the predetermined areas in the exceptionmanagement table 4002T. For example, the area name of the area 401 ofFIG. 34A is BaseBar, the area name of the area 402 of FIG. 34B isShell_TrayWnd, and the area name of the area 403 of FIG. 34C istooltip_class32.

Alternatively, identification information to uniquely identify the areamay be registered in the exception management table 4002T instead of thearea name. For example, each area has not only an area name but also anumber (which may include numeric values and signs) which is assigned bythe manufacturer of the OS. Thus, each area may also be identified bythis number as the identification information.

FIG. 37 is a flowchart for explaining a process in which the displaydata acquiring unit 451 acquires a whole area display data or a specificarea display data. The process of FIG. 37 is similar to that of FIG. 23,and only primary steps in the process of FIG. 37 which differ fromcorresponding steps in the process of FIG. 23 will be described. It isassumed that in the process of FIG. 37, the user aa has already clickedthe share setting input button 301 shown in FIG. 21B.

First, the operation input receiving unit 46 determines whetherselection of the whole area is received (S9051). When the selection ofthe whole area is received (YES in S9051), the display data acquiringunit 451 acquires the display data of the whole desktop screen displayedon the display 216 (S9052), and the process of FIG. 37 is terminated.

When the selection of the whole area is not received (NO in S9051), thedisplay data acquiring unit 451 determines whether an active area ispresent (S9053). When the active area is present (YES in S9053), thedisplay data acquiring unit 451 acquires the area name of the activearea from the OS and compares the acquired area name with each of thearea names registered in the exception management table 4002T (S9053-1).When coincidence does not occur, the acquired area name is compared withall the area names of the exception management table 4002T. Whencoincidence occurs, the comparison is stopped.

When the area name of the active area is registered in the exceptionmanagement table 4002T (YES in S9053-2), the display data acquiring unit451 acquires the display data of the whole desktop screen displayed onthe display 216 (S9052), and the process of FIG. 37 is terminated. Thus,even if the area, such as BaseBar 401 shown in FIG. 34A, is changed tothe active area, the display data of the whole area may be displayedwithout displaying the display data of the active area. Alternatively,instead of displaying the display data of the whole area, the displaydata of the area, whose area name is not registered in the exceptionmanagement table 4002T, which was active immediately before may beacquired.

When the area name of the active area is not registered in the exceptionmanagement table 4002T (NO in S9053-2), the display data of the areawhich is active among the display data currently displayed on thedisplay 216 by the display control unit are acquired (S9054). Namely, inthis case, the display data of the active area are acquired similar toEmbodiment 1. Subsequent steps are the same as those corresponding stepsof Embodiment 1, and continuously the process of FIG. 22 is performed.

The process of FIG. 37 provides a modification of the process of FIG. 23of Embodiment 1. Also, the processing of steps S9053-1 and S9053-2 inthe process of FIG. 37 may be similarly to the process of FIG. 29 ofEmbodiment 2. Thus, regardless of how the user selects the active area,it is possible to prevent the display data of the areas registered inthe exception management table 4002T from being shared.

Also in Embodiment 4, the icon of the mouse cursor may be incorporatedin the display data of the active area similar to Embodiment 3.

In the display data acquisition program 1451 installed in the externalinput device 40, a predetermined menu is prepared to stop the comparisonof the acquired area name of the active area with the area namesregistered in the exception management table 4002T. If the user desiresto share the display data of the area registered in the exceptionmanagement table 4002T, the setting of the sharing may be selected fromthe predetermined menu. Thus, the user may arbitrarily select permissionor inhibition of sharing of the display data of the area the area nameof which is registered in the exception management table 4002T.

As described above, according to Embodiment 4, it is possible to preventacquisition of the display data of the area which are not to beacquired. Thus, it is possible to reduce problems, such as frequentchanges of the display data by operations of the user.

Embodiment 5

In Embodiments 1, 2 and 4, the display data of the active area and thewhole area which are arbitrarily selected by the user may be shared bythe terminals. However, when the external input device receives theactive area or the whole area from the viewing-area selection menu 310(FIG. 28A) displayed on the desktop screen as in Embodiment 2, a problemmay arise. Specifically, the user may select one of the radio buttons302,303 of the viewing-area selection menu 310 when the viewing-areaselection menu 310 is incorporated as one application window. Theviewing-area selection menu 310 may become an active area and thedisplay data thereof may be shared by the terminals. As a result, such asituation that the participants of the videoconference viewing thedisplay data on the display 120 have a sense of incongruity in regardsto the display data may arise.

In Embodiment 5, the transmission system in which, when the viewing-areaselection menu 310 incorporated as a window changes to an active area byuser's operation, the sharing of the display data of the viewing-areaselection menu 310 is inhibited will be described. Specifically, whenone of the radio buttons 302,303 of the viewing-area selection menu 310is selected, the display data acquiring unit 451 is configured to accessthe exception management DB 4002 and not to acquire the display dataeven if the window of the viewing-area selection menu 310 is active.Thus, when one of the radio buttons 302,303 is selected, it is possibleto prevent the display data of the viewing-area selection menu 310 frombeing shared.

FIG. 38 is a diagram showing an exception management table 4002T1 ofEmbodiment 5. The functional configuration of the external input device40 may be the same as that of FIG. 35 of Embodiment 4. The area names(window names) of the areas the display data of which are not to beacquired even if the areas become active are registered in the exceptionmanagement table 4002T1. In this embodiment, an area name “RScreenShare”is additionally registered in the exception management table 4002T1.“RScreenShare” is the area name of the viewing-area selection menu 310incorporated as a window. Thus, the display data acquiring unit 451 canprevent sharing of the display data when the viewing-area selection menu310 incorporated as a window becomes active.

FIG. 39 is a flowchart for explaining a process in which the displaydata acquiring unit 451 acquires a whole area display data or a specificarea display data. The process of FIG. 39 is similar to that of FIG. 37,and only primary steps in the process of FIG. 39 which differ fromcorresponding steps in the process of FIG. 37 will be described.

The user aa depresses or clicks the share setting input button 301“start sharing” as shown in FIG. 28A. As a result, the window of theviewing-area selection menu 310 becomes active. At the time ofdepression, either of the radio buttons 302 and 303 may be selected.There are the case where “whole area (full screen)” is selected and thecase where “active area” is selected.

The processing of steps S9051, S9052, S9053, and S9053-1 in FIG. 39 isthe same as that of corresponding steps in FIG. 37, and the explanationthereof is omitted.

In the process of FIG. 39, when the area name of the active area isregistered in the exception management table 4002T1 (YES in S9053-2) andan initiation flag is not equal to “1” (NO in S9053-3), the display dataacquiring unit 451 does not acquire the display data of the wholedesktop screen. The initiation flag is a flag which is set to “1” whenthe programs for the external input devices are started and reset to “0”when the whole area display data are acquired. The initiation flag isprovided for controlling the behavior of the programs immediately afterstarting of the programs for the external input devices. In the processof FIG. 37 (Embodiment 4), when the area name of the active areacoincides with the area name registered in the exception managementtable, the display data of the whole area (full screen) are shared. Inthis embodiment, however, the display data of the whole area are notshared. Exceptionally, in this embodiment, the display data of the wholearea are shared immediately after starting of the programs for theexternal input devices.

When the area name of the active area is not registered in the exceptionmanagement table 4002T1 (NO in S9053-2), the display data of the areawhich is active among the display data currently displayed on thedisplay 216 by the display control unit are acquired (S9054). Namely,the display data of the active area are acquired similar to Embodiment4.

Therefore, the external input device 40 of this embodiment operates asfollows.

First, immediately after the programs for the external input devices arestarted, the viewing-area selection menu 310 of FIG. 28A is in aninitial state where the radio button 302 is selected.

(i) When the share setting input button 301 is pressed while the “wholearea (full screen)” of the radio button 302 is selected, the result ofthe determination at S9051 is YES, and the display data acquiring unit451 starts sharing of the whole area display data.

(ii) When the share setting input button 301 is pressed while the“active area” of the radio button 303 is selected by the user, thewindow of the viewing-area selection menu 310 becomes active. Sharing ofthe display data of the viewing-area selection menu 310 is notappropriate and the active area is not present, and the programs for theexternal input devices may share the display data of the whole area. Toavoid the problem, the initiation flag is utilized.

When the radio button 303 is selected, it is determined at step S9053that the active area (the viewing-area selection menu 310) is present,and the result of the determination at step S9053-2 is YES. Immediatelyafter the programs for the external input devices are started, it isdetermined at step S9053-3 that the initiation flag is equal to “1”, andthe display data acquiring unit 451 acquires the display data of thewhole area at step S9052. After the display data of the whole area areacquired, the initiation flag is reset to “0”.

Namely, immediately after the sharing of the display data is started,sharing of the display data of the whole area (full screen) is startedeven if either of the radio buttons 302 and 303 is selected. Thereafter,if the user activates the area which is not registered in the exceptionmanagement table 4002T1, the display data of the area which is activatedby the user are shared.

(iii) When the share setting input button 301 is pressed while thedisplay data of the “whole area (full screen)” are shared and the “wholearea (full screen)” of the radio button 302 is selected, the result ofthe determination at step S9051 is YES. Thus, the display data acquiringunit 451 continuously performs the sharing of the display data of thewhole area.

When the share setting input button 301 is pressed while the displaydata of the “whole area (full screen)” are shared and the “active area”of the radio button 303 is selected, the result of the determination atstep S9053-2 is YES and the result of the determination at step S9053-3is NO. Thus, the display data acquiring unit 451 continuously performsthe sharing of the display data of the whole area.

Namely, even if either of the radio buttons 302 and 303 are selected,the sharing of the display data of the whole area is continued. Thedisplay data acquiring unit 451 shares the display data of the activearea in the state where the display data of the “whole area (fullscreen)” is shared, only in a case where the user selects (or activates)the area of the area name which is not registered in the exceptionmanagement table 4002T1.

(iv) When the share setting input button 301 is pressed while thedisplay data of the “active area” are shared and the “whole area (fullscreen)” of the radio button 302 is selected, the result of thedetermination at step S9051 is YES. Thus, the display data acquiringunit 451 starts sharing of the display data of the whole area.

When the share setting input button 301 is pressed while the displaydata of the “active area” are shared and the “active area” of the radiobutton 303 is selected, the result of the determination at step S9053-2is YES and the result of the determination at step S9053-3 is No. Thus,the display data acquiring unit 451 continues sharing of the displaydata of the active area which was active before the viewing-areaselection menu 310 becomes active.

Namely, even if either of the radio buttons 302 and 303 are selected,the viewing-area selection menu 310 becomes active. In this case, whenthe “active area” of the radio button 303 is selected, sharing of thedisplay data of the area which was active immediately before iscontinued, and when the “whole area (full screen)” of the radio button302 is selected, sharing of the display data of the whole area isstarted.

According to the external input device 40 of this embodiment, theprevious state of sharing of the display data is held even if the windowincluding the viewing-area selection menu 310 which are not suitable forsharing of the display data is selected. One of the sharing of the wholearea display data and the sharing of the active area display data may beswitched to the other without giving sense of incongruity to theparticipants of the videoconference.

As described in the foregoing, according to the invention, it ispossible to provide a transmission system in which a user on atransmission terminal may select arbitrarily one of sharing of wholearea display data and sharing of specific area display data such thatthe selected display data are shared with a communication partner of avideoconference on the other transmission terminal.

The transmission system according to the invention is not limited to theabove-described embodiments and various variations and modifications maybe made without departing from the scope of the invention.

The present application is based on and claims the benefit of priorityof Japanese Patent Application No. 2012-250979, filed on Nov. 15, 2012,Japanese Patent Application No. 2013-064085, filed on Mar. 26, 2013, andJapanese Patent Application No. 2013-135200, filed on Jun. 27, 2013, theentire contents of which are hereby incorporated by reference.

1. A non-transitory computer-readable recording medium storing a programrepresenting a sequence of instructions, the program which when executedby a computer, causes the computer to perform a process, the computerconnected to a first transmission terminal which is connected to a firstdisplay device and connected to a second transmission terminal via anetwork, the process comprising: displaying an area of a programcomponent on a screen of a second display device connected to thecomputer; receiving selection of at least one of display data displayedin a whole area on the screen of the second display device and displaydata displayed in the area of the program component on the screen as anoperation target of a user; acquiring the display data of the whole areaor the area indicated by the received selection; receiving firstresolution information indicating a resolution of the first displaydevice and transmitted by the first transmission terminal; determining aresolution of the acquired display data based on the received firstresolution information and second resolution information indicating aresolution of the second display device; and transmitting the acquireddisplay data with the determined resolution to the first transmissionterminal.
 2. The non-transitory computer-readable recording mediumaccording to claim 1, wherein an operating system is executed on thecomputer to treat the whole area and the area as separate programcomponents and manage at least position information of the programcomponents, and the receiving the selection comprises: receiving aposition on the screen where an event occurs associated with a pointingdevice from the operating system; receiving, when the received positionoverlaps the whole area different from the area of the programcomponent, selection of the display data of the whole area; andreceiving, when the received position does not overlap the whole area,selection of the display data of the area of the program component asthe operation target of the user.
 3. The non-transitorycomputer-readable recording medium according to claim 1, wherein thereceiving the selection comprises: displaying on the second displaydevice a selection menu used to receive selection of one of the wholearea and the area of the program component as the operation target ofthe user on the screen; receiving selection of the display data of thewhole area when the whole area in the selection menu is selected; andreceiving selection of the display data of the area of the programcomponent when the area of the program component as the operation targetof the user in the selection menu is selected.
 4. The non-transitorycomputer-readable recording medium according to claim 1, wherein theprocess further comprises: generating, when the display data of the areaof the program component as the operation target of the user arereceived in the acquiring the display data and a resolution of thereceived display data is less than a threshold, display data having aresolution greater than the threshold by adding arbitrary pixel valuesto the received display data.
 5. The non-transitory computer-readablerecording medium according to claim 1, wherein the acquiring the displaydata comprises acquiring the display data of the whole area when theselection of the display data of the area of the program component asthe operation target of the user is received in the receiving theselection and an area name of the area of the program component as theoperation target of the user is identical to that registered in an areaname management table.
 6. The non-transitory computer-readable recordingmedium according to claim 1, wherein the acquiring the display datacomprises continuing acquisition of the display data of the whole areawhen the selection of the display data of the area of the programcomponent as the operation target of the user is received in thereceiving the selection and an area name of the area of the programcomponent as the operation target of the user is identical to thatregistered in an area name management table.
 7. The non-transitorycomputer-readable recording medium according to claim 5, wherein thearea name management table includes at least area names of areasregistered therein, the area names of the areas used in the receivingthe selection of the at least one of the display data of the whole areaand the display data of the area of the program component as theoperation target of the user.
 8. The non-transitory computer-readablerecording medium according to claim 1, wherein the process furthercomprises: displaying on the second display device a display datatransmission permission menu used to receive permission or inhibition ofsharing of the acquired display data by the first and secondtransmission terminals; starting the transmission of the acquireddisplay data with the determined resolution to the first transmissionterminal when the permission is received from the display datatransmission permission menu; and stopping the transmission of thedetermined display data with the determined resolution to the firsttransmission terminal when the second transmission terminal startstransmission of display data or when the inhibition is received from thedisplay data transmission permission menu.
 9. The non-transitorycomputer-readable recording medium according to claim 1, wherein theacquiring the display data comprises: acquiring cursor informationcontaining coordinate data of a cursor that is moved on the screen ofthe second display device and interlocked with a pointing device;incorporating, when selection of the display data of the whole area isreceived, an image of the cursor in the display data at a positionindicated by the coordinate data; and incorporating, when selection ofthe display data of the area of the program component as the operationtarget of the user is received, the image of the cursor in the displaydata at a position indicated by the coordinate data of the cursor whichare compensated by coordinate data of the area of the program componenton the screen.
 10. The non-transitory computer-readable recording mediumaccording to claim 9, wherein the cursor information contains thecoordinate data and image data of the cursor, and the acquiring thecursor information comprises acquiring the coordinate data and the imagedata of the cursor.
 11. The non-transitory computer-readable recordingmedium according to claim 9, wherein the cursor is a mouse cursor.
 12. Atransmission system including: a first transmission terminal which isconnected to a first display device and connected to a secondtransmission terminal via a network; and an external input device whichis connected to the first transmission terminal and transmits displaydata to the first transmission terminal, the first transmission terminalcomprising: a resolution acquisition unit configured to acquire aresolution that is usable by the first display device; and a resolutioninformation transmitting unit configured to transmit first resolutioninformation indicating the resolution to the external input device, theexternal input device comprising: a display control unit configured todisplay an area of a program component on a screen of a second displaydevice connected to the external input device; an operation inputreceiving unit configured to receive selection of at least one ofdisplay data displayed in a whole area on the screen of the seconddisplay device and display data displayed in the area of the programcomponent on the screen as an operation target of a user; a display dataacquiring unit configured to acquire the display data of the whole areaor the area of the program component indicated by the receivedselection; a resolution information receiving unit configured to receivethe first resolution information from the first transmission terminal; aresolution converting unit configured to determine a resolution of theacquired display data based on the received first resolution informationand second resolution information indicating a resolution of the seconddisplay device; and a display data transmitting unit configured totransmit the acquired display data with the resolution determined by theresolution converting unit to the first transmission terminal.
 13. Thetransmission system according to claim 12, wherein an operating systemis executed on the external input device to treat the whole area and thearea of the program component as separate program components and manageat least position information of the program components, and theoperation input receiving unit is configured to receive a position onthe screen where an event occurs associated with a pointing device fromthe operating system, receive, when the received position overlaps theprogram component of the whole area different from the program componentof the area, selection of the display data of the whole area, andreceive, when the received position does not overlap the programcomponent of the whole area, selection of the display data of the areaof the program component as the operation target of the user.
 14. Thetransmission system according to claim 12, wherein the operation inputreceiving unit is configured to display on the second display device aselection menu used to receive selection of one of the whole area andthe area of the program component as the operation target of the user onthe screen, receive selection of the display data of the whole area whenthe whole area in the selection menu is selected, and receive selectionof the display data of the area of the program component when the areaof the program component as the operation target of the user in theselection menu is selected.